81

After reading this article, I realized that there is something fundamentally wrong with how web site owners are using CloudFlare as their CDN/firewall/DDOS/DNS/everything-under-the-hood solution.

A lot of my daily troubleshooting searching eventually ends up at Stack Overflow or one of its sister sites, and I always get prompted for the dreaded reCAPTCHA page. This has been happening for about a year.

I thought there was something wrong at my end or maybe everyone else in the world is also facing the same issue, but this article made me realize that the problem is not me, my IP address, or my ISP; the problem is with CloudFlare and how they overzealously handle their so called firewall settings.

In the past, I sent a couple of emails to Stack Overflow about this issue, without any luck. Is there any chance that something will be done to fix this problem?

EDIT: Just to update. There seems to be some confusion and assumption about this question, and it's mostly because I was not very clear with some details. Sorry about that.

Some facts in an attempt to clear out any confusion.

  • I live in South East Asia (SEA).
  • I don't use Tor, VPN, proxy or any crazy browser extension that affects my browsing activity.
  • The problem is not only limited to Stack Exchange sites; it happens to any site that is using CloudFlare as their DNS/DDOS solution
  • I have emailed CloudFlare/Stack Exchange and other website owners (using CloudFlare) in the past, without getting any permanent solution.
  • I go to google.com, search for a problem I need a solution for. I get a Stack Overflow link on my Google search result. I get excited, and I click the link, and then I get prompted for a CAPTCHA. It's been like this since Stack Exchange started using CloudFlare (which I don't know when). So overnight a perfectly working website (or group of websites under Stack Exchange), started being an annoyance to use. With no way for me to fix it.
  • I posted here mostly out of frustration and because I read the above mentioned post I linked.
  • My Stack Overflow writing skills are not that great; I might have missed some hidden rules, so excuse me for that. My Stack Overflow activity usually limits to reading with very occasional few writing. This is the most I am been active in an single question.

Thanks for all the suggestion and comments.

27
  • 35
    You always get prompted when you do what exactly? View a page, post a comment, post a question/answer... Please add some more details.
    – Laurel
    Commented May 21, 2016 at 22:16
  • 4
    When search for answers on google and follow links from there. Doesnt happen when I directly go to stackoverflow.
    – bran
    Commented May 21, 2016 at 22:35
  • 8
    I don't remember ever getting one by following a link. I did just get one between the time I posted that last comment and now, (and I had to find all the pictures of store fronts to post my answer), but it's not "ruining" anything for me. Now I must ask: what do you need to do, find the pictures or just check the box?
    – Laurel
    Commented May 21, 2016 at 22:40
  • 4
    Sounds like to me that it blocks Google's referrer that your specific localized Google uses. Does the problem happen if you use google.com?
    – AStopher
    Commented May 22, 2016 at 19:04
  • 3
    'web site owners are using CloudFlare as their <...>DDOS/DNS<...>' whaaat? Dude, CloudFlare is used to protect websites from (D)DOS ((Distributed) Denial Of Service) attacks.
    – ForceBru
    Commented May 22, 2016 at 20:03
  • 2
    @ForceBru in this context I was referring them as solutions, "ie DDOS/dns solutions". I can understand if the wording wasn't clear enough.
    – bran
    Commented May 22, 2016 at 21:04
  • 44
    I've been here for years and I don't remember ever having to go through a CAPTCHA to browse here, either coming directly to the site or arriving through links. I never have any CAPTCHA problems, even when I am not signed in.
    – Geeky Guy
    Commented May 23, 2016 at 12:03
  • 7
    As a Tor user, I suffer this every day in a lot of sites. MathJax is affected too, so math markup in math.stackexchange.com and similar often doesn't show up. Commented May 23, 2016 at 13:26
  • 1
    Are you using SO at work when this happens, and behind a corporate proxy? cloudflare probably sees your using a proxy and rates it as more risk than a direct connection. I dont have issues at work connecting to SO with our proxy but google always locks me out as our proxy shows im from japan when google knows my regular login location as london Commented May 23, 2016 at 13:35
  • 2
    "Doesnt happen when I directly go to stackoverflow." If that's true, may give a HTTP referer header hiding extension a try. Here's one: addons.mozilla.org/en-US/firefox/addon/smart-referer Commented May 24, 2016 at 3:10
  • 16
    @Renan "Works on my machine" has 21 upvotes?? Commented May 24, 2016 at 6:25
  • 7
    @Millie Smith I feel like most people didnt even bother to read the original post. Maybe peole dont know that SE uses cloudflare as their CDN?
    – bran
    Commented May 24, 2016 at 6:36
  • 4
    @MillieSmith I think it's more like 21 other people are also not having this problem. This is useful information because it helps to find out the exact cause of this CloudFlare related problem. We now know it may have to do with Tor, for example, so anyone coming through a machine that has ever been used as a node knows why they are being threated like bots wherever they go (I think CloudFlare is wrong on doing that, BTW).
    – Geeky Guy
    Commented May 24, 2016 at 11:54
  • 1
    @MillieSmith I upvoted that comment because I've never had a recaptcha for any SO/SE site on any of my devices, from a variety of networks. Yes, I read the original post, bran, but my upvoting that comment doesn't mean I think you're imagining it. Rather, as Renan said, it seems potentially useful data to help determine the scope of this problem.
    – Beofett
    Commented May 24, 2016 at 14:19
  • 1
    Indeed. As Beofett said. I upvoted Renans post because it makes more sense to upvote his 32 times than for 32 people to make a "me too" post.
    – Knossos
    Commented May 24, 2016 at 14:49

3 Answers 3

52

According to this comment by a SE employee they are going to stop using Cloudflare as their CDN/DDOS solution soon so it should be fixed from next month:

I am on the SRE team for Stack Exchange/Stack Overflow which is mentioned in the article. We are a paying customer of CloudFlare but have had a lot of issues with 503 errors (edge cannot reach our origin servers) occurring in various regions on a monthly basis. That combined with a few other issues (DNS change delays, missed deadlines, limiting tor users) has lead us to look for another CDN provider.

A few on our team still use their services for personal blogs and projects, but we just signed a contract with a new CDN provider and plan on switching off CloudFlare for our network of sites in June.

-- by /u/gbrayut

2
  • 1
    This is said employee, by the way
    – Quill
    Commented May 26, 2016 at 1:36
  • 7
    We are now off CloudFlare and running on Fastly - making this as [status-completed].
    – Nick Craver Mod
    Commented Jul 1, 2016 at 14:55
17

Tor has an article about CloudFlare's problems; briefly:

  • CloudFlare appears to never un-block/un-soft-block IP addresses that have ever been used for spam (or, perhaps, other malicious purposes). Given the nature of Tor, this means that upwards of 80% of Tor exit nodes have already been "tainted", and the percentage is increasing steadily over time.
  • Akamai's stats on conversion rates suggests legitimate e-commerce on Tor is of similar prevalence to the web as a whole, so CloudFlare's aggressive blocking is unwarranted.

There is a way for sites (such as SO) to whitelist access over Tor.

1
  • 1
    This explains a lot about at least part of why using Tor is such an awful end-user experience. Commented May 24, 2016 at 18:47
13

I found this JavaScript GIST that will rewrite Google search results.

https://gist.github.com/nikcub/3720482

My recommendation is to rewrite StackOverflow URLs to use HTTPS.

I can only hope that this will help differentiate your IP address from those attacking port 80, and also help preserve your cookies.

The script looks like it needs some work, but the idea is sound. Force SO links to HTTPS and see if that helps.

Note: HTTPS is not officially supported by SO, but hey it works for me.

11
  • 49
    Alternatively, HTTPS Everywhere is a plugin that does this for a lot of sites, including SO apparently. Commented May 22, 2016 at 18:46
  • 5
    I use HTTPS Everywhere; it works on SO, with a caveat: the dynamic updates as other people enter answers etc, and the automatic aging of time links on questions, etc, doesn't work — everything is in stasis unless you hit refresh. I still use HTTPS Everywhere, despite that. Commented May 23, 2016 at 0:57
  • 5
    @JonathanLeffler I use HTTPS Everywhere and the live updates work fine.
    – Navin
    Commented May 23, 2016 at 2:14
  • 3
    @JonathanLeffler Also on HTTPS Everywhere here, live updates all work just fine for me.
    – user229044 Mod
    Commented May 23, 2016 at 2:56
  • @Meagar: Intriguing. I wonder what it is that's inhibiting mine? Let's see: Mac OS X 10.11.5, Firefox 46.0.1. Extensions: ABP, Ghostery, NoScript might be guilty — any one of them, or some combination. However, there are no blocked sites per NoScript's icon. I'm open to ideas — I'd love to get the 'live updates' back after being without them for most of a year, IIRC. Commented May 23, 2016 at 3:03
  • 4
    @JonathanLeffler corporate proxy servers are notorious for screwing up web-socket traffic - on http (ws://) they can make a lot of mess and corrupt the stream by trying to treat it as http when it isn't; on https (wss://) there are proxies that just drop the connections because they aren't expecting them to stay open. Is it possible that you're behind a misbehaving proxy?
    – Marc Gravell Mod
    Commented May 23, 2016 at 8:22
  • @MarcGravell: It is possible… I'm at home, with a wireless connection to an Apple 'AirPort Time Capsule', which connects to a Comcast Business cable modem, which connects to 'the internet'. I also see the same behaviour at the office, where I use a wireless connection to the corporate network. Commented May 23, 2016 at 13:49
  • @MarcGravell That's certainly not my issue, same issue on home network and not. When rules enabled, I always see this in the console, and as such web-socket related updates fail: "Firefox can't establish a connection to the server at wss://qa.sockets.stackexchange.com/." Issue on GitHub: github.com/EFForg/https-everywhere/issues/3884 Commented May 23, 2016 at 16:51
  • https works for all main SE sites. It doesn't work for any per-site metas (excluding meta.AU, meta.SF, meta.SU, and meta.SO). Also, you might get a few broken links...
    – Zizouz212
    Commented May 23, 2016 at 22:19
  • Thatnks for the siggestion. I feel that since the problem is not at my end and the problem happens to sizeavle number of people. And the problem is something tied to websites using cloudflare, either cloudflare or SE should be the one trying to find a fix not me.
    – bran
    Commented May 24, 2016 at 6:45
  • @JonathanLeffler: there's a feature in NoScript to force SSL on selected sites, maybe that will help?
    – halfer
    Commented May 24, 2016 at 9:59

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .