9

I already wrote email to the Stack Overflow team but that doesn't seem to help. Robot answering?

For some odd reason (and from time to time) when opening stackoverflow.com I'm getting blocked with the "Too many requests" message. It happens for a few days, then it's gone, to show up a week later or so.

When I do a netstat -an | grep 151.101 I can see a ton of request are being setup when using Safari (just opening a single page), when doing the same with Chrome it seems to work fine.

Anybody got a clue what is going on?

tcp4       0      0  10.0.1.36.63684        151.101.65.69.80       ESTABLISHED
tcp4       0      0  10.0.1.36.63647        151.101.193.69.443     ESTABLISHED
tcp4       0      0  10.0.1.36.63646        151.101.65.69.80       ESTABLISHED
tcp4       0      0  10.0.1.36.63644        151.101.65.69.80       ESTABLISHED
tcp4       0      0  10.0.1.36.63631        151.101.65.69.443      ESTABLISHED
tcp4       0      0  10.0.1.36.63630        151.101.65.69.80       ESTABLISHED
tcp4       0      0  10.0.1.36.63649        151.101.65.69.80       TIME_WAIT  
tcp4       0      0  10.0.1.36.63651        151.101.65.69.80       TIME_WAIT  
tcp4       0      0  10.0.1.36.63652        151.101.65.69.80       TIME_WAIT  
tcp4       0      0  10.0.1.36.63653        151.101.65.69.80       TIME_WAIT  
tcp4       0      0  10.0.1.36.63655        151.101.65.69.80       TIME_WAIT  
tcp4       0      0  10.0.1.36.63657        151.101.65.69.80       TIME_WAIT  
tcp4       0      0  10.0.1.36.63658        151.101.65.69.80       TIME_WAIT  
tcp4       0      0  10.0.1.36.63659        151.101.65.69.80       TIME_WAIT  
tcp4       0      0  10.0.1.36.63660        151.101.65.69.80       TIME_WAIT  
tcp4       0      0  10.0.1.36.63661        151.101.65.69.80       TIME_WAIT  
tcp4       0      0  10.0.1.36.63663        151.101.65.69.80       TIME_WAIT  
tcp4       0      0  10.0.1.36.63664        151.101.65.69.80       TIME_WAIT  
tcp4       0      0  10.0.1.36.63665        151.101.65.69.80       TIME_WAIT  
tcp4       0      0  10.0.1.36.63666        151.101.65.69.80       TIME_WAIT  
tcp4       0      0  10.0.1.36.63667        151.101.65.69.80       TIME_WAIT  
tcp4       0      0  10.0.1.36.63668        151.101.65.69.80       TIME_WAIT  
tcp4       0      0  10.0.1.36.63669        151.101.65.69.80       TIME_WAIT  
tcp4       0      0  10.0.1.36.63670        151.101.65.69.80       TIME_WAIT  
tcp4       0      0  10.0.1.36.63671        151.101.65.69.80       TIME_WAIT  
tcp4       0      0  10.0.1.36.63672        151.101.65.69.80       TIME_WAIT  
tcp4       0      0  10.0.1.36.63674        151.101.65.69.80       TIME_WAIT  
tcp4       0      0  10.0.1.36.63675        151.101.65.69.80       TIME_WAIT  
tcp4       0      0  10.0.1.36.63676        151.101.65.69.80       TIME_WAIT  
tcp4       0      0  10.0.1.36.63677        151.101.65.69.80       TIME_WAIT  
tcp4       0      0  10.0.1.36.63678        151.101.65.69.80       TIME_WAIT  
tcp4       0      0  10.0.1.36.63679        151.101.65.69.80       TIME_WAIT  
tcp4       0      0  10.0.1.36.63680        151.101.65.69.80       TIME_WAIT  
tcp4       0      0  10.0.1.36.63681        151.101.65.69.80       TIME_WAIT  
tcp4       0      0  10.0.1.36.63683        151.101.65.69.80       TIME_WAIT  
4
  • We did send out a roboty standard message to a bunch of people a couple days ago, because we had a huge backlog of these rate limit tickets due to a problem that never got corrected. We simply had no idea which of the tickets in that long list actually got solved by the change which was made, or which ones would still exhibit problems. So the entire queue got the same message with an invitation to write back if they were still having problems because we anticipated it would hit some users who it didn't help.
    – animuson StaffMod
    Commented Mar 30, 2017 at 17:02
  • @animuson alright, got ya. Did you guys fix it? It seems to work again! Thanks!
    – Roger
    Commented Mar 30, 2017 at 18:41
  • Nothing that would have fixed this particular issue. You're not the only person who has been rate limited because of Apple sending these bulk requests. It's... not something we can do much about. We don't really want to increase limits so that a faulty browser can continue making a bunch of bad requests every time a user loads a page, which is effectively tripling the number of incoming requests for some users.
    – animuson StaffMod
    Commented Mar 31, 2017 at 15:16
  • That netstat output is kind of weird. Why does it format ip:port in that fashion? Commented Mar 31, 2017 at 15:48

1 Answer 1

9

We're seeing a pretty large number of requests for apple-touch-icon.png (which exists, albeit as a redirect) and apple-touch-icon-precomposed.png (which does not exist).

I've no idea what's triggering these (the names of the images and your statement that you're using Safari suggests some form of Apple madness), but in some cases we're seeing multiple requests per second, which'll trigger the rate-limits pretty quickly.

There are also a lot of requests for the Stack Overflow homepage, which may be what's triggering the image requests. Again, sometimes more than 1 per second; can't explain that either, but if by chance you're sitting there hitting "refresh" as fast as you can... Might wanna not do that.

8
  • 1
    @Shog0 yeah i know this is what Safari does (ran into that myself with some webapps). However, it doesn't explain why this only a few days, then its gone, and then its back again. Are you guys replying with a 404 ? BTW apple is pulling in a ton of request for different sizes i think. developer.apple.com/library/content/documentation/… (sigh)
    – Roger
    Commented Mar 29, 2017 at 18:59
  • 404 for the one that doesn't exist, 302 for the one that does (redirects to CDN).
    – Shog9
    Commented Mar 29, 2017 at 19:27
  • I wonder if this is related to the new version of Safari that just came out. I'm suddenly having trouble getting to the SO home page and the updated Safari is the only thing that's changed at this end (that I can think of).
    – matt
    Commented Mar 31, 2017 at 15:15
  • Would removing <link rel="apple-touch-icon" ...> or adding <link rel="apple-touch-icon-precomposed" ...> (and on CDN too) to home page have an impact?
    – Cœur
    Commented Mar 31, 2017 at 15:25
  • @Cœur Removing that would make it worse. The problem is the browser is attempting to load http://stackoverflow.com/apple-touch-icon.png, but we host it on our CDN, which that <link> is meant to point out so they don't need to make that pointless request. Removing it would likely cause even more users to start sending these redirected requests.
    – animuson StaffMod
    Commented Mar 31, 2017 at 15:35
  • 2
    display a fullscreen banner "this site is optimized for Chrome, Firefox and Opera: please use a real browser" :D
    – Cœur
    Commented Mar 31, 2017 at 15:40
  • sometimes more than 1 per second - how many more, and are the in clusters? For example, maybe the favicon for a bunch of saved bookmarks?
    – Izkata
    Commented Mar 31, 2017 at 15:47
  • Honestly just looks like safari is requesting the homepage + favicons repeatedly, for minutes at a time @Izkata. No idea why.
    – Shog9
    Commented Mar 31, 2017 at 18:22

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .