85

I wanted to extract and refine an interesting element of πάντα ῥεῖ's recent question as an individual feature request:

When a new user's first n posts (n=3? 4?) all contain the same URL, that should raise an automatic system flag for moderators to look into. I believe this would be an effective indicator of excessive self-promotion and potentially spam.

I'm not talking about an automatic spam flag or a more severe action against a user, just a system flag of the style we get when a user posts an identical answer across multiple questions. A new user posting the same link in multiple places seems like something moderators should check out, even if it's something like someone promoting their blog or a GitHub project. In the latter cases, we might want to caution them before they trigger a strong allergic response from the users here.

Again, we already get system flags for users deleting too much of their own content, editing too much of their stuff, and posting duplicate answers. I think a flag like this would have a higher signal-to-noise ratio than those, and could catch spammers slipping through review (like our recent Toolbox spammers). It should also be reasonably simple to check, only testing for the same URL in the first n posts of a user.

14
  • 29
    I'd think we would want this network wide, not just for SO - maybe this should be on MSE? :)
    – Taryn Mod
    Commented Sep 18, 2014 at 15:53
  • 5
    Don't spammers alternate the urls that are posted though to prevent being caught by systems just like this? I still upvoted this since it does catch the people who are in it for the self-advertisement, or post unnecessary signatures in all their stuff.
    – Gimby
    Commented Sep 18, 2014 at 16:03
  • i'm surprised that this feature doesn't exist already. new users posting the same like oodles of times seems iffy Commented Sep 18, 2014 at 16:08
  • 14
    My gut is telling me we should check for the same URL in the last n posts by any user, but I'd want to see some data to see if it makes any difference.
    – Bill the Lizard Mod
    Commented Sep 18, 2014 at 16:13
  • 11
    @Gimby - the clever spammers might, but most spammers aren't clever.
    – ChrisF Mod
    Commented Sep 18, 2014 at 16:14
  • I recently flagged an answerer who posted his blog's link as answers in many posts, as well as in the past using 'others'. An automated system would be very useful because this behaviour of new users will be detected early and they can be made aware about the rules of the site. Commented Sep 18, 2014 at 16:31
  • 1
    "In the latter cases, we might want to caution them before they trigger a strong allergic response from the users here." By the time they post their first answer, it's already too late. You'll be finding robo-reviewers BSODing and and organic reviewers having seizures all over the place.
    – BoltClock Mod
    Commented Sep 18, 2014 at 17:18
  • 16
    What about not triggering this on links to known reference sites? I frequently post a link to, say, api.jquery.com, so that people can know I'm getting my information from an authoritative source. Commented Sep 18, 2014 at 17:48
  • @MikeMcCaughan I think that would be great, I normally post a JavaDoc link (docs.oracle.com/...) under every answer for users/visitors that want to know more detail
    – msrd0
    Commented Sep 18, 2014 at 18:09
  • 1
    Good idea, but we should 'white list' some URLs. For example, lxr.free-electrons.com/source is very often used to cite pieces of the Linux kernel.
    – Peter L.
    Commented Sep 18, 2014 at 18:12
  • 12
    @MikeMcCaughan - What are the odds that the first 3 or 4 posts of a legitimate user all point to the same reference site? I don't imagine this will be a common occurrence, and moderators can easily dismiss the flags if this happens. All the flag does is bring attention to odd behavior, not anything else. The vast majority of the system-triggered potential vandalism flags are false positives, and I can deal with a handful of these if it reduces the effort required by the developers to build and maintain a whitelist.
    – Brad Larson Mod
    Commented Sep 18, 2014 at 18:25
  • @BradLarson Good points. You're right that maintaining the white list would be pain. I know MSDN, for example, has changed their URL structure several times over the years. Commented Sep 18, 2014 at 18:56
  • @MikeMcCaughan The whitelist maintenance could be automated. For example, a back-end script could parse answers (and maybe comments) that have positive feedback and look for URLs. Those URLs could then be programmatically upvoted into a whitelist, or at least their net name (e.g., lxr.free-elecrons.com, api.jquery.com, etc.).
    – Peter L.
    Commented Sep 18, 2014 at 23:38
  • What the system can't do without robot-crawling URLs can be done by the community: if a user gets quite downvoted or has a lot of spam flags on his answers/questions and a high number of posts containing the same or different URLs, that's probably a sweet spot. In the worst case the system will trigger a warning for a terrible user posting continuous garbage supported by random documentation links (and thus very eligible for the "low quality posts" queue). +1 for this warning. Shouldn't be terribly hard to implement either (I hope)
    – Marco A.
    Commented Sep 19, 2014 at 3:56

3 Answers 3

23

I love this idea. As Gimby mentioned in the comments, more advanced spammers could still get around this by using multiple URLs, but it should help prevent the less sophisticated spammers that Gimby mentioned (self promotors, signatures that contain URLs) from getting through.

I think that anything we can do to make spammers up their game is a worthwhile effort, especially for something like this that seems like it should be a fairly easy thing to implement.

11

In addition to identifying accounts that repeatedly post the exact same link, I think it would be useful to also identify domains that have been linked repeatedly (say, n > 2) by only a single account. That could help identify people repeatedly linking to their own site without requiring white-listing of common documentation sites to keep false positives low.

4

There are a couple different user cases here that should be looked at.

The crafty spammer

This is the one people seem to be pointing at first and saying it won't catch them. This is true. The system currently in place won't catch them either. That is, there is no system in place to catch crafty spammers who play with urls to try to avoid pragmatic detection. Thats something that we, the readers do and notice and say "hey wait a moment..." and start throwing spam flags around.

The well meaning false positive

Yes, people linking to google searches, MSDN, w3schools, wikipedia, grepcode, lxr.free-electrons.com or other useful sources of canonical information. They would trigger such a flag. There are two ways to deal with this (either, or both in combination):

  • Global whitelists maintained by the mods. Some domain name list that the mods say "yep, thats a good source, don't bother me with it."
  • A per user 'ok' list. A bit more user oriented, a mod has gotten an auto-flag for a given user posting links to some resource. The mod looks, says 'ok', and it gets marked as ok.

  • The I didn't know I was not supposed to do that

This is the ones that the suggestion is trying to catch. People trying to boost their page rank or just publicize their blog a little bit through some answers. The most notorious forms of this are the half an answer as an answer here (hey, its an answer, its immune to the NAA flag) with a link for "more information can be found at XYZ.com"). Kind of like those Super Bowl ads for Victoria's secrets and go-daddy (not that I watch those ads).

Sometimes this is again a well meaning person who doesn't know that the community tends to get rather irate when this is discovered and shows up with down votes en masse. For someone who is trying to follow the rules, a bit of a warning from a mod might avoid a user who would otherwise be quite put off when the meta effect hits hard. This is what the system is supposed to catch.

Other times, this is someone who is actually trying to spam (and just not doing a great job of it - like the one that this proposal draws from). The person is essentially putting a signature file with their blog or consultancy in every post. And again, this is what the proposal is supposed to try to catch. In this case, the mod would likely be a bit more slap on the wrist and say "cut it out" before it becomes a real problem.

The other ones? Its going to be a little bit of overhead that gets cleaned up quickly for the mods, or won't show up there any more than it does now with the occasional custom flag.

On that 'per user ok list' I mentioned

Just random late night brainstorming. Each time a person puts a link in a post that link is associated with the user. This information is kind of already out there - you can search for specific urls via advanced search (example: url:"*.pastebin.com" (and yes, thats a different search than url:"pastebin.com"). I'm not sure how much is actually behind that search.

With some heuristic known by the devs, when that counter goes over a certain threshold for a given number of posts, the auto-flag is raised and the mods can look at it. The reason I mentioned that per-user ok, is that one could hypothetically have another boolean field associated with that counter of 'ok' which would permanently exclude that url from raising additional flags for that user.

6
  • Little did the spammers know that SO automatically adds rel="nofollow" so it won't help their page rank at all.
    – KyleMit StaffMod
    Commented Sep 20, 2014 at 3:41
  • 2
    @KyleMit given the suggested edits I've seen elsewhere on the site, some don't care. The others are after the click through from SO (even if they don't understand why it own't help their page rank). However, IIRC, with sufficient votes, the nofollow does disappear (see What is the threshold for removing rel=nofollow from links in posts?). You can see this if you poke around the source of highly scored answers with outbound links.
    – user289086
    Commented Sep 20, 2014 at 5:06
  • It does not help that new users can not leave comments, but can quickly find a few questions when a comment linking to their blog is the correct outcome. (Assume someone search with google before writing the blog post and found related questions.) Commented Sep 20, 2014 at 15:46
  • @IanRingrose if you can find an example where someone has written enough in the blog that wouldn't be an acceptable as a post, I would be most interested in them. Granted, they'd likely be 10k links, but screen shots would work too. An answer of "look at my blog over there" isn't acceptable for anyone - 1 rep or 100k rep.
    – user289086
    Commented Sep 21, 2014 at 0:48
  • @MichaelT, but it can be acceptable as a comment, yet we don't let new users leave comments Commented Sep 21, 2014 at 21:04
  • @IanRingrose if a user has sufficient material that is in a blog post (their own or other) and is able to understand the material well enough, they are able to write an answer to the question that would be sufficient to stand alone as an answer. Linking to their blog in a comment, be it a 1 rep user or a 100k is not a good use of a comment. With the Q&A format, answers are here - not over there. I would likely flag a comment that says "I've answered this in my blog post over there" as spam.
    – user289086
    Commented Sep 21, 2014 at 22:35

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .