English English French French Spanish Spanish German German
Geekflare is supported by our audience. We may earn affiliate commissions from buying links on this site.
Share on:

Complete Guide To Detect AI Chatbot Plagiarism

Complete Guide To Detect AI Chatbot Plagiarism
Invicti Web Application Security Scanner – the only solution that delivers automatic verification of vulnerabilities with Proof-Based Scanning™.

AI Chatbots like ChatGPT aren’t limited to just answering your questions. They can write human-like emails, cover letters, essays, poetry, blog posts, and a lot more.

AI writing is like a chameleon – it can blend in with any other type of writing so well that you’d be hard-pressed to tell the difference.

Unfortunately, this makes it incredibly difficult for us humans to detect when an AI has written something instead of a person. Sure, we could all take Sherlock Holmes’ advice and “eliminate the impossible,” but who has time for that?

What if I tell you that the above paragraph is written by an AI? Yes, I asked YouWrite to write a quick introduction for this post, which is what it came up with. Pretty convincing right? That’s the problem, it’s too convincing that anyone can present it as their own work to benefit from it.

AIs don’t just rephrase the content; they learn the topic and present it in a natural language using RLHF. This means the generated content will neither feel robotic to read nor can it be compared with data available online to catch plagiarism.

Catching AI-generated content isn’t easy, and I am not going to give you false hope that you can catch it with 100% accuracy. However, with a little bit of detective work and some help from AI writing detection tools, you can catch most works of AI.

Below you’ll find manual ways to detect AI plagiarism and some automatic tools too.

Manually Detect AI Plagiarism 🕵️

Manually-Detect-AI-Plagiarism

ChatGPT isn’t even a big problem when it comes to AI plagiarism. There are many alternative tools that are based on the same GPT-3 technology and are built for writing. These are very advanced and specifically made to create human-like content that is difficult to catch.

Automatic detection tools are a bit hit or miss depending on content type, length, and complexity. So at least currently, manually connecting the dots to figure out whether the write-up is AI-generated or not is the best way.

Below I have listed some common clues that you could look for to catch AI-generated content.

#1. Outdated Information

ChatGPT and other GPT-3 based AI had their knowledge cut at the end of 2021. Even though some AI bots have a built-in search engine to look for the latest information, they still rely on old data to write long-form content like essays or blog posts.

If you notice the write-up constantly talking about outdated data, specifically before 2021, then it could be AI-generated.

In the below screenshot, I asked YouWrite to tell me about the recent FIFA World Cup, and this is what it came up with.

Outdated-Information

It talks about the World Cup 2018 instead of the recent 2022 one. Even though the tool is updated to talk about recent events like the 2022 World Cup, for writing long posts, it still uses the outdated data it is built on.

#2. Uses the Same Sentence Structure for Descriptions

While writing descriptions of multiple products/apps, AI usually uses the same sentence structure that it edits to suit the description of a product. The descriptions usually start with the same word, like The, It, A, or the product’s name. The descriptions also have a dedicated focus that it repeats.

For example, below, I asked ChatGPT to tell me about “data entry jobs websites”. You can see all descriptions start with “A” and has a strict focus.

Uses-the-Same-Sentence-Structure-for-Descriptions

I also asked YouChat the same question, and it did the same thing and repeated the name of each website at the start, and the descriptions themselves look like a rephrased version of each.

YouChat

If the write-up you are analyzing has a description of anything similar in nature, it’s a great way to catch AI writing.

#3. Uses Short Sentences

For most write-ups, AI will use one-liner sentences with a single comma at most. Maybe they try to minimize the chance of error, but they avoid writing long sentences that might require the use of colons, semicolons, or heavy horizontal lines.

I know this isn’t the best clue to catch AI writing, but it’s worth considering to further prove your suspicion.

#4. Can provide incorrect information

For complex questions that usually involve some kind of instruction, the AI can fail to understand the intention and provide incorrect information. I am not talking about outdated information here, they just confidently tell you the wrong information when they don’t grasp the topic.

For example, I asked YouWrite to tell me “how to root an Android phone without a PC”. It still showed me methods that involve the use of a PC at some point in instruction.

Can-provide-incorrect-information

#5. Doesn’t Add a Personal Opinion

When talking about something, AI just talks about known facts. Unlike humans, it won’t talk about its personal experience with it. Although AIs can be instructed to create a hands-on write-up, they tend to avoid doing it by default.

If you notice the write-up is only talking about well-known facts, it’s a good chance it’s written by an AI. For example, while talking about an app, it will talk about its features instead of how smooth the navigation is or how actually to use the app.

#6. Brief Explanations

As AIs are limited by the knowledge currently available, they tend to write brief explanations and descriptions. In my experience, if they talk about a particular subject — such as Android rooting — they keep the explanation within 2-3 paragraphs max. And for things like apps or products, the description usually doesn’t exceed 1 paragraph.

Humans can write in-depth about something, including features, limitations, personal experience, pricing, etc. On the other hand, AI usually focuses on the main features or highlights, so the write-up is short too. If the instructor asks for more details, the chance of incorrect information significantly increases.

#7. Check the writer’s history

As a teacher or webmaster, if you have access to the writer’s previous work or assignments, it might be a good idea to compare it with the write-up you are suspicious about. It’s highly unlikely that the writer has a similar writing style to an AI. Even though AI can change writing tones, but styles it chooses are rather generic, and it can’t copy your tone.

Taking the example of the AI-generated paragraph in the intro of this post. Anyone who has read my previous posts can easily see that my writing style is entirely different compared to it.

You can just read 2-3 previous works of the writer and see if it matches the writing style of the suspicious write-up.

Tools To Detect AI Plagiarism

Tools-To-Detect-AI-Plagiarism

There are a bunch of tools available that can detect AI-written content. However, their accuracy is different from each other and can fail depending on the type of content and length.

I tested these tools using different types of content generated by ChatGPT, GPT-3 Playground, WriteSonic, Rytr, and YouWrite. They managed to detect all types of content generated by ChatGPT and GPT-3 Playground.

Although they had mixed results for dedicated writing tools, interestingly, they all failed to detect YouWrite; I am guessing it’s because YouWrite makes human-like mistakes on purposes to avoid detection.

I will recommend you use these tools alongside manually checking the content. First, analyze the content with one of these tools and then manually check for any clues to confirm the suspicion further.

Below I am listing the ones that offered the best results and are the easiest to use:

Note: For demonstration, I am going to use generated text from WriteSonic on “Benefits of Mangoes”.

#1. Content At Scale

I found the AI content detector by Content At Scale to be the most accurate for most of the tests I ran. It lets you scan up to 2500 characters per scan and makes a percentage prediction showing real or fake content density.

Content At Scale

Content that is over 60% fake is safe to assume is written by AI content, at least most of it. In the below screenshot, you can see says 96% content is fake, which I fully generated with AI.

#2. GPTZero

If you want to see exactly what content is AI generated, then GPTZero is a good option. GPTZero doesn’t score content. Instead, it highlights the exact content that is probably generated by AI.

GPTZero

Although it does show perplexity and burstiness scores to get an idea of randomness in the text, more importantly, it supports 5000 characters per scan and can also upload files. Scanning large content greatly helps in improving accuracy.

GPTZero

#3. GPT-2 Output Detector

The creators of ChatGPT also offer a bot to detect their own work. As GPT-2 and GPT-3 are only different in the total data used for training, this GPT-2 detector works fine for GPT-3 based AIs too.

GPT-2-Output-Detector

There is no limit on how much content it can scan, so it’s perfect for detecting very long AI-generated content. I also found its accuracy fine, especially for ChatGPT-generated content. However, it’s a bit lose in predictions, so only consider its results when it shows over 50% of the content as fake.

#4. Writer AI Content Detector

Writer AI Content Detector results are questionable when detecting content from AI writing tools, but it works great for ChatGPT-generated content. You can scan 1500 characters per scan and even add the URL of the content if it’s already published.

Writer AI Detector

For my test AI-generated text, it said 94% of the content is human-generated, which is quite off compared to other tools. Nonetheless, it’s still worth using to detect ChatGPT content, especially if it’s already published.

#5. Draft & Goal

A simple yet powerful AI writing detection tool. In my experience, Draft & Goal offered similar results to Content At Scale, but it doesn’t have a word count limit so that you can scan long write-ups. However, its scan time increases based on how much content you scan.

Draft-Goal

For my example text, it says that 94% of the text is AI-generated.

My Thoughts 💭

I personally think AI help is actually a great way to remove writer’s block, get all the information in one place, and find inspiration. However, it should never replace human writing as AI is limited to the knowledge already known.

You may also explore some best plagiarism checking tools to prevent copying.

Thanks to our Sponsors
More great readings on AI
Power Your Business
Some of the tools and services to help your business grow.
  • Invicti uses the Proof-Based Scanning™ to automatically verify the identified vulnerabilities and generate actionable results within just hours.
    Try Invicti
  • Web scraping, residential proxy, proxy manager, web unlocker, search engine crawler, and all you need to collect web data.
    Try Brightdata
  • Semrush is an all-in-one digital marketing solution with more than 50 tools in SEO, social media, and content marketing.
    Try Semrush
  • Intruder is an online vulnerability scanner that finds cyber security weaknesses in your infrastructure, to avoid costly data breaches.
    Try Intruder