44

I saw a hot meta post the other day about plagiarism on Documents, so I decided to write a bot to help automate the process of handling plagiarism quickly and easily.

It's a user script which autopilots the process of scanning an entire tag, or every tag, one document at a time, 30 words at a time to quickly pinpoint and handle plagiarism on the new Documents site.

It autopilots your browser, opening Documents pages, opening Google, performing Google searches in verbatim mode, using fuzzy string searches on the top results to determine plagiarism, communicating back to the Documents tab with that info, and then consulting you before taking an action on any pinpointed plagiarized content.

I can compile a list of sources which allow themselves to be cited for documentation, so that if the bot finds un-cited plagiarism of one of those sources, it can put it in a quote format and cite the original source. Or the bot can flag it. Or it can remove it and cite the original source in the edit summary.

Regardless, the user makes the call, so they can always deviate from the suggested action. I just need to know which action the bot should suggest under which circumstances.

12
  • 8
    Oh hey, this sounds really cool. Might be useful for checking out plagiarists' other contributions in the Q&A section of the site (if it works). Definitely keep us posted!
    – Pekka
    Commented Aug 2, 2016 at 22:07
  • 11
    @Pekka웃 I'll open source it on Github when the current features are done pretty soon and we can expand it to handle general SO posts.
    – user5536767
    Commented Aug 2, 2016 at 22:10
  • @Pekka웃 I'd rather not bother Shog with such a trivial question and I think you may have a better idea than me: At what percentage of a post being made up of cited 3rd party sources does it near the threshold of #1 or #3 in Shog's answer? 10%? 20%? More? Below this threshold the bot will simply add citations to the sources, but above this threshold, it'll suggest flagging.
    – user5536767
    Commented Aug 3, 2016 at 15:57
  • that's a good question, not trivial at all. I don't have a quick answer - one'd have to look at some examples to get an inkling. Do you mean by "cited 3rd party sources" ones that acknowledge their sources, or don't?
    – Pekka
    Commented Aug 3, 2016 at 16:14
  • @Pekka웃 well the idea is that the bot can easily cite them, so you can consider them all cited. Its just a matter of "when is there too much cited content?" More advanced parameters could be used than just a flat percentage as well if needed, such as ratio of original code to cited content, or some other parameters.
    – user5536767
    Commented Aug 3, 2016 at 16:29
  • 1
    right. Personally, I'd start at 20% and see what happens (= if there are too many false positives). This is really cool, looking forward to testing it
    – Pekka
    Commented Aug 3, 2016 at 16:39
  • @Pekka웃 Any ideas? I've got all of the code roughly complete, however I hit a snag on communicating between Google and SO. Luckily, I think I see a possibility in the GM API which allows us to do a xmlhttpRequest completely ignoring cross origin policies, which lets us load Google into an iFrame on SO, but then all the scripts and sources are pointing to SO as the domain. I'm trying to figure out a way to trick the iFrame contents to think the domain is google.com.
    – user5536767
    Commented Aug 4, 2016 at 4:19
  • I have no better ideas than what was already said I'm afraid. If you ever need bounty support let me know
    – Pekka
    Commented Aug 4, 2016 at 8:30
  • This is a good tool for Documentation where information is not meant to be duplicated. On the main site, I find it highly implausible that the tool could be used effectively except for blatant cases of plagiarism. There's already plenty of answer styles that are essentially plagiarism but considered acceptable either with light attribution or because it's "well known" knowledge. If everybody had to source their posts, the amount of low quality sources that will show up in posts would skyrocket. Commented Aug 4, 2016 at 21:39
  • 1
    @Pekka웃 Don't worry, it wasn't a dead end. Besides this potential solution I figured out another: I can open the google page with the correct search query in the URL, handle the search on Google, then from Google open the SO homepage as a proxy, including the result as a query in the URL and have that page read the query and send the result to the already-open documentation page without being obstructed by same origin policy. :)
    – user5536767
    Commented Aug 5, 2016 at 1:48
  • @Pekka웃 Sorry to bother you again, but if you don't mind I would take you up on that bounty support offer since this question has been open for some time and I haven't been able to discover a solution. If I use my 100 rep as a bounty, I lose commenting and chat room privileges right? If I keep the privileges, I'll use my rep. This problem alone has been keeping me from finishing the anti plagiarism script.
    – user5536767
    Commented Aug 10, 2016 at 23:18
  • @uhohsomebodyneedsapupper That's a fair point. I'm not a very experienced SO user, so I'll leave that discussion to the experts.
    – user5536767
    Commented Aug 10, 2016 at 23:24

2 Answers 2

34
  1. I think it's safe to say we never want entire topics or examples to be copied verbatim. If you run across that, flag it and/or just delete it.

  2. Things get a bit dicey when there's a small amount of a larger example that was ripped from another place without due credit given. Sometimes these are ok: a sentence or two explaining something in a way that augments the rest of the post, or a short code sample that illustrates the idiomatic way to do something. Editing to give credit results in a useful example.

  3. ...Then there are those examples whose authors appear to have cobbled them together from numerous sources as though they were creating a ransom note. These can be harder to detect... And harder still to distinguish from #2. I've been tricked into thinking someone had just borrowed a bit of phrasing, only to realize that the entire post was drawn from about 3 different sources, mixed together with maybe a sentence of the author's own words. If you do manage to find and sort one of these out, please flag and explain what's going on...

  4. Finally, there are the sections that started out as plagiarism and then went through a dozen or so edits, leaving something that resembles the original but differs in significant ways. Adding attribution may no longer be possible, but wholesale deletion throws away the work of later editors - which can be substantial. If you're not comfortable deleting, I recommend editing in a note to the effect of, "adapted from [source]"... And then flagging so we can keep an eye on the shameless original author.

Don't forget to report back and let us know how your script is working out!

5
  • 15
    What does it say about me that I'm savvy enough to detect instances of #3 on sight, with a decent (> 80%) success rate?
    – BoltClock Mod
    Commented Aug 3, 2016 at 3:23
  • 18
    @bolt It says that most people's "own words" are so crummy that you can assume anything eloquent was probably copy-pasted. I have a similar skill set. I'm not sure if that actually makes you feel any better....
    – Cody Gray Mod
    Commented Aug 3, 2016 at 9:59
  • @Cody Gray: Well, it means either I'm not crazy, or I'm not the only crazy one...
    – BoltClock Mod
    Commented Aug 5, 2016 at 5:08
  • @BoltClock it's just a trivial application of Sturgeon's law on a global scale. Commented Aug 5, 2016 at 12:31
  • 2
    @CodyGray hey! Where did you get the word "eloquent" from? ;-) Commented Aug 5, 2016 at 12:31
10

Nice, you actually did it! (Side question, are you using any frameworks or did you program that from scratch?)

Unfortunately, Documentation doesn't yet have the tools for handling plagiarism in an ideal way, even when doing it by hand. Flagging on Docs is not flagging for real, rolling back rolls too much, and it can get complicated.

I've been answering a bit about this already, so hopefully that might help you out.

I'd also like to mention that it sometimes does require looking at context a bit in order to know the real story.

In one case, an edit by a 3 rep user alerted me to the presence of copied content from a 200k+ rep user. It was only after a bit of digging that I figured out that the 200k+ user was likely the original author reposting their content to Docs, but I'm still not sure what the correct action is. We need some clearer guidelines on this.

I have also had hits that go the other way, despite how new Docs is. Fortunately, I was paying attention that the result was a meta post discussing (and quoting) the Documentation, which means it wasn't plagiarism.

In any case, I would read up on what proper attribution involves.

2
  • 10
    Then again, it feels wrong to say from scratch as a modern day developer, with all the layers of abstraction other people built between us and the processor. Standing on the shoulders of giants is right.
    – user5536767
    Commented Aug 3, 2016 at 2:06
  • 1
    Everyone, upvote for jQuery.
    – tbodt
    Commented Aug 3, 2016 at 22:35