216

The site xspdf.com keeps showing on my search list every time I'm searching for programming or data science-related questions. It's rated higher than Stack Overflow even though the content is just a generated PDF style of content. This site provides misleading answers and is nothing else than a badly formatted Q&A site (never provides anything useful).

My question: Why does this site keep showing up on search engine results at the top of the list (multiple answers) and how can it be blocked because is annoying and why is Stack Overflow or search engine is not enforcing it?

I would expect Stack Overflow to block it since xspdf.com is a company that sells products and this type of content is just generating traffic for them (I see them as fraud).

Search engine list Claiming that is content from Stack Overflow Content from Stack Overflow

Updated to include details:

Search query used: “pyspark dataframe columns to vector”

The Google engine results does show 3rd and 4th in the top list first page (phone browser):

Google search engine

Using the DuckDuckGo search engine (phone browser) doesn’t provide any trace of this xspdf.com site:

Top 3 results

and

More content on results.

VPN with restricted mode, no account and no history on VPN using Google Chrome and the Google search engine:

Results using VPN on work machine

Incognito with all security enabled and strict mode as allowed by the Mozilla Firefox browser using the Google search engine:

Query and search incognito

Exact same results as above (strict, incognito, trace removal, containerization of services, anything of what I were able to think of in order to secure the browser and search and even ranking higher than the inventors of Spark:

Exact same results as above (strict, incognito, trace removal, containerization of services, anything of what I were able to think of in order to secure the browser and search

Note: It is clearly tricking the Google search engine part of NLU, based on the structure of the site to get “weight” in the search queries.

———-

Adding information from the site (it looks a scam and is not a reputable site to use Stack Overflow content to sell their product or trick users:

Home page of the site

Their customers seems to be very happy ex US presidents as top users, so it seems to be a government-loved product (jokes aside):

Happy users

Based on answers I feel that a note/clarification will help here: Is not the problem to add a “filter” to query, install a plugin/extension, run a script in browser to “exclude” what is nonsense, the whole point of the question here is “what tools or options do we have available to prevent this behavior in the future” or “how legit sites with TRUE content can gain priority on search engines and copy-paste ones to be on the “desired list” ?

License does state to provide source but not just a master name as can be seen in screens posted (feels like research papers should include as source just the name of the journal and not actual referenced paper as the source).

In the end: I do feel that stackoverflow should get priority on search engines and not a “proxy” or “bot” site. This site is not the only one but is one of the top that doesn’t have any content listed or searchable for the users but is rated by engine as top content (can’t even get to a list of articles/posts there).

26
  • 81
    I have also noticed that xspdf.com plagues the search results, sometimes in the first 3-4 results. I am wondering how they manage to trick the Google crawler into making it so relevant with next to zero original content. Commented Jan 23, 2021 at 8:13
  • 40
    I admit I'm not a legal expert but the content on stackoverflow is licensed under CC-BY-SA and your last screenshot shows that they have provided attribution and the license specification. Legally speaking, I don't think they did nothing wrong.
    – LuxGiammi
    Commented Jan 23, 2021 at 11:34
  • 18
  • 25
    Perhaps a result of search results personalization? I've never seen this site and SO often comes on top for myself. Try searching for the same terms in incognito and see if xspdf still appears so high.
    – mck
    Commented Jan 23, 2021 at 14:02
  • 6
    My first result on private browsing when I type "for loop for each dataframe column " on Google is Stack Overflow, and I have no sign of xspdf on the first page either. Is it maybe because Google registered that you have visited that website a lot? Edit: Basically what @mck said just above.
    – Clockwork
    Commented Jan 23, 2021 at 15:11
  • 2
    What is the exact search terms you are using? It is truncated in the screen shot. Would be interesting to see if we get the same Commented Jan 23, 2021 at 17:13
  • 1
    @mck Initially tough too but all my browsers are restrictive, no cookies, no scripts, no tracking, even on VPN on work machine I do get this site (initially were under impression that was part of company push but seems is just a “cheating” method to “trick” google engine ... I started to see it once google implemented bert models and statistically is correct the content but is nothing else than a copy site. Other sites uses SO content but at least does put effort to keep original SO content with added/improved formatting but this site is nothing else than a bot to generate “clogged” content.
    – n1tk
    Commented Jan 23, 2021 at 18:06
  • 3
    So can you give us an example search term that returns this site as a top result? Commented Jan 23, 2021 at 18:09
  • 1
    @MartinSmith, updated question to include search query and details (from phone and posting from VPN work results too)
    – n1tk
    Commented Jan 23, 2021 at 18:26
  • 3
    @Clockwork adding "pyspark" to the end of that search term I see it on the first page (for loop for each dataframe column pyspark) I guess they are ranked high in searches for that technology which is why some people are plagued by it and others have never seen it Commented Jan 23, 2021 at 18:37
  • 2
    @MartinSmith Yeah, I just tried, same result, 4th one. Although it doesn't seem to appear at all when I try on Bing and DuckDuckGo. I guess only Google is plagued.
    – Clockwork
    Commented Jan 23, 2021 at 18:59
  • 5
    Nope but is the best to “fool” a reputable engine like google ... is not about license but the content generated to “throw” weight of an engine to put on top of the search without having anything original in it ...
    – n1tk
    Commented Jan 23, 2021 at 22:39
  • 3
    @LuxGiammi I think it's illegal, because no link and name of creator as required for CC-BY-SA Commented Jan 24, 2021 at 10:31
  • 2
    Oh my goodness. It appears in every stackoverflow result.... Commented Jan 24, 2021 at 17:39
  • 4
    I'm pretty sure author attribution is required, not just attribution to SO. Search results, No author attribution on this page, but they do mention Stack Overflow, Screenshot of no author attribution in case they fix it later
    – jrh
    Commented Jan 24, 2021 at 19:11

4 Answers 4

86

You can report it to Google: https://www.google.com/webmasters/tools/spamreportform

3
  • This is a good suggestion, just not sure how quick google is in reacting to it. Till then personal blocklist it is. Commented Feb 4, 2021 at 14:14
  • apparently nothing yet happened as of today.
    – seedme
    Commented Feb 22, 2021 at 21:59
  • @seedme When I search for pyspark dataframe columns to vectors now I don't see this xspdf.com site anywhere in the results anymore 🎉
    – ruohola
    Commented Apr 22, 2021 at 10:08
41

There are two ways that I know to filter sites out of your Google Search results.

  1. Append -site:xspdf.com to your query
  2. Use a browser extension like Personal Blocklist (Chrome, Firefox)
4
  • 23
    Those are fixing the symptoms rather than the cause, having to install a 3rd party browser extension that only is available for some of the browsers, or to type some extra query string to each search, is not optimal.
    – Cristik
    Commented Jan 23, 2021 at 17:05
  • 47
    @Cristik I agree. However, hacker1024 is not an employee of SE Inc. Temporary solutions/workarounds are always welcome, in my opinion.
    – 41686d6564
    Commented Jan 23, 2021 at 17:10
  • 1
    On personal machine can be done but is decrease productivity and on phone is kinda impossible to do it with easy + adding extensions is not always an easy solution to push security team to enable into large corporations ...
    – n1tk
    Commented Jan 23, 2021 at 18:46
  • 7
    Rather than manually adding it to every search, most browsers should allow it to be added in the search engine query string in the search engine settings page.
    – hacker1024
    Commented Jan 24, 2021 at 12:18
22

I would expect Stack Overflow to block it

Anyone is free to use Stack Overflow's content under the license. It follows that, anyone is free to make a Stack Overflow clone with existing content, as long as they follow the terms of the license.

I suspect Google will soon get the hint that their top result is getting preferentially ignored, and soon things will fix themselves. Who knows, perhaps PDF matches in authoritative-looking PDFs are getting bumped by the algorithm.

4
19

While this isn't a fix to the underlying issue, if you just want to see answers from Stack Overflow, you can add "site:stackoverflow.com" to your search query. This is similar to hacker1024's answer, except that it specifically limits your results to only stackoverflow.com, rather than specifically excluding xspdf.com from the results.

The "site:x" filter is quite useful when you want to search a particular site that doesn't have a search feature built in or whose built-in search feature doesn't have a very good search algorithm behind it. Good thing we don't know any sites like that. ;)

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .