121

I've written a very long answer, with a lot of details - Now, I want to add even more, but, when I pasted the latest version of my answer into the edit box, it told me that answers are not allowed to be longer than 30k characters:

Screenshot

Some details about my answer:

  • It's a list of things that can be done in order prevent or deter something (website scraping).

  • There's very little code, and most of the text isn't code.

Some possibilites I've come up with:

  1. Post a longer version of my answer somewhere else, and link to it - however this would mean more effort to maintain two different versions, and I don't really like this idea.

  2. Try to make it fit into 30k characters - this would be quite some effort, and not a really a long-term solution, as I will inevitably want to add to it.

  3. Post it somewhere else (for example, a blog), and delete it from Stack Overflow - I don't really like this solution, since I love Stack Overflow, I don't have a blog, and I've got a feeling that nobody will ever see it and benefit from it. (And I wouldn't get reputation points for it ;-))

  4. Just leave it as is - However I don't really like the idea leaving it as is when I could improve it (in fact, I already have improved it).

  5. Just continue in a second post.

So, what should I do? Is this kind of answer even appropriate for Stack Overflow?

This is the answer: How do I prevent site scraping?

(I've also put the latest version on GitHub, which is around 40k characters).

As a bit of an update: I've now done what Joe and others have suggested: I've trimmed down the answer, removed a lot of fluff and some of the less important points, and I've added a link to the GitHub page with the more complete version. Thank you all for your input and opinion, and I now know a lot more about Stack Overflow than I did before :-)

45
  • 184
    Maybe this is the epitome of "too broad" - "good answers would be too long"?
    – Makoto
    Commented Feb 2, 2016 at 15:39
  • 4
    The question itself is too long to capture here, so that's what my gut is telling me. Perhaps if it were broken out into more distinct and digestible pieces, it could be acceptable...
    – Makoto
    Commented Feb 2, 2016 at 15:42
  • 3
    There are a few cases where an answer is split into multiple separate posts just to fit all the minute detail in. But that's rarely appropriate. Commented Feb 2, 2016 at 15:44
  • 63
    30k characters is too long for an answer in my book, full stop.
    – Joe
    Commented Feb 2, 2016 at 15:44
  • 50
    Ive just read your answer, its good but there is a LOT of blurb. I'm going to go out on a limb and say I think you could keep the useful content, remove the blurb and easily fit in the 30K limit
    – Jamiec
    Commented Feb 2, 2016 at 15:54
  • 3
    I think it is a good reference answer; however I find the longer one on GitHub not so good, as it has become repetitious and not so concise and directed. In this case the word limit has helped the focus. Commented Feb 2, 2016 at 15:57
  • 3
    WOW! I would not dare - for sure the question would be deleted just before I posted my humongous, detailed answer:( Commented Feb 2, 2016 at 16:37
  • 11
    That would actually make a decent book, then you could sell it and make real money off of it instead of fake internet points.
    – user4639281
    Commented Feb 2, 2016 at 17:12
  • 3
    @Tiny, I actually often find it annoying when there's huge amounts of code / logs / traces in the question itself - I guess the character limit was specifically implemented to reduce that kind of thing. I think that the best way to include such things in a question is to include what seems / is relevant in the question, and the rest as a linked gist / pastebin - although others may disagree with me. Commented Feb 2, 2016 at 17:35
  • 3
    @JonasCz you should never create two posts/answers on StackOverflow. This disrupts the possibility of a "best" answer, as then both of your answers would be best.
    – user3373470
    Commented Feb 2, 2016 at 18:20
  • 9
    @onebree You might want to qualify that. Posting multiple answers is fine if they're all distinct answers; posting one answer as a continuation of another answer just to get around the length limit is not okay. Commented Feb 2, 2016 at 19:59
  • 22
    "I have made this longer than usual because I have not had time to make it shorter." — Blaise Pascal; "If a sentence does not illuminate your subject in some new and useful way, scratch it out" — Kurt Vonnegut, Jr. IEEE Transactions on Professional Communication.
    – msw
    Commented Feb 2, 2016 at 19:59
  • 3
    I never imagined that anybody would ever hit the limit. I always knew about it, but I've always ignored it. Commented Feb 2, 2016 at 20:25
  • 5
    @UiDesigner Incomplete answers deserve downvotes. Also, can you imagine how confusing that would be if the answers were not in order? Users can change their sorting preferences, you know. Commented Feb 2, 2016 at 20:36
  • 4
    @PeterMortensen Re that Electronics.SE example: That's actually kind of horrible IMO - The "addition" is actually accepted and pinned to the top, thus fragmenting the answer and requiring extra effort to understand. Also, it's the huge amount of code which is causing that answer to go over the length limit, an that's a different problem really. I recently answered a similar question here with a link to a specific class in my OSS project which did exactly what the OP wanted, plus a description of how I implemented it - I think this is a lot nicer than dumping a 1k line class into an answer. Commented Feb 2, 2016 at 21:13

8 Answers 8

209

Answers that are 30k characters long are too long for this site. The site doesn't handle them well - you have to scroll through the entire answer to get to anything below it, and that's really long. Stack Overflow also doesn't aim for questions/answers that require 30k character answers - short, reasonably well-tailored questions/answers are the aim.

Feeling that a question needs an answer of 30k+ characters tells me one of two things. Either the question is too broad in scope, or the answer is too broad in scope for the question.

In your case, it's a bit of both I think: the question asked is very broad; but you also are writing a very broad answer which attempts to answer everything possible related to the question. That's not necessarily a bad thing, per se, but for a question that's already this broad, it means you have too large of an answer for this site.

So, what should you do?

Write a blog post

It sounds like you want to share a lot of information on this topic. Great! Write a blog post with all of the information you have there. Then, summarize it in a Stack Overflow answer, and link to the post for more information. If the answer, by itself, is a good answer, then this is perfect: you have the necessary information in the answer, plus more information in the post.

Narrow your answer's scope

Instead of trying to answer all possible things remotely related to the question, pick a narrower scope. You don't have to give a full background and comprehensive list of things to look at; you can focus on the main points. What is one major approach? Or a couple?

Limit the detail in your answer

If you do want to provide a 'breadth' answer, then remove some of the fine details. Your section headers look like they're pretty descriptive and actionable; maybe they don't need all that text under them. This isn't always as good of an answer as a narrower but deeper answer, but when accompanying a bunch of other answers (as yours is) sometimes it's a better contribution overall.

16
  • 7
    This looks good - Less detail in the answer itself, with a more detailed version elsewhere, and a link to it in the answer. Looks like that's what I will probably do. Since I don't have a blog (yet) I might as well stick with keeping it on GitHub. Commented Feb 2, 2016 at 16:05
  • 6
    Another possibility which comes up pretty rarely: if there are multiple divergent answers bundled into a single answer it would be appropriate to answer the question multiple times. I don't feel that multiple answers is appropriate in this specific case, but I think it's worth at least mentioning with a disclaimer.
    – zzzzBov
    Commented Feb 2, 2016 at 16:44
  • 18
    This answer is 2000 chars. His is 15x longer than this!
    – usr
    Commented Feb 2, 2016 at 17:11
  • 2
    @JonasCz it's fairly trivial to set up GitHub and GitHub pages for a static blog / HTML (or using a blog engine like Jekyll)
    – ashes999
    Commented Feb 2, 2016 at 17:25
  • 1
    @JonasCz It may be a better idea to use github gists, as your current repo only holds one file. If you have not done so already, please edit your answer. Instead of detailing what each heading means, scrap all that and just put the headings into a list. Although not the same format, this answer of mine pointed to a LONG gist, with too much code for SO Q&A. And if you ARE interested in a blog, maybe put your answer in multiple posts. This is a great starter entry/series :-)
    – user3373470
    Commented Feb 2, 2016 at 18:15
  • 1
    @onebree, GH gists don't have issues or pull requests (Unlikely that i'll need those, but who knows), and I might add more files to that repo (code snippets, images, license...) But yes, otherwise that's an interesting idea, although I generally like to have things on SO itself instead of a short explanation and a link for more. Commented Feb 2, 2016 at 18:38
  • @usr 18.5x even (37321 characters). The limit is 15 times this answer.
    – Mixxiphoid
    Commented Feb 2, 2016 at 19:57
  • I don't have a blog Perhaps look for a popular blogger that writes about a related topic and ask if you could contribute a guest post.
    – Eric J.
    Commented Feb 2, 2016 at 22:08
  • 2
    Why isn't there a Stack Overflow blog, I wonder? Other SE sites have them.
    – Joe
    Commented Feb 2, 2016 at 22:09
  • @Joe because it became "the Stack Overflow©" blog
    – Braiam
    Commented Feb 2, 2016 at 22:18
  • The comment "Answers that are 30k characters long are too long for this site." may be true but seems to draw a bit of an arbitrary line. Who came up with 30k? What about 29k? Etc..
    – Tommy
    Commented Feb 3, 2016 at 3:25
  • 1
    @Tommy Jeff, most likely. And what's wrong with arbitrary? Got to draw it somewhere.
    – Joe
    Commented Feb 3, 2016 at 3:26
  • @Joe: 'Why isn't there a Stack Overflow blog, I wonder?' Ironically, according to this: blog.stackoverflow.com/2012/05/encyclopedia-stack-exchange ,SO is also the blog..
    – GitaarLAB
    Commented Feb 3, 2016 at 14:14
  • 1
    That answer is way too long for a blog too. Better with multiple blog posts, part 1, part 2....
    – Abhitalks
    Commented Feb 4, 2016 at 11:32
  • 1
    @Tommy Everyone can have opinion but think of the alternative: no limits at all. It doesn't sound very viable option IMHO. The guy who pays the bills, wears the biggest hat or is in charge for some other reason, has only two options: to limit or not to limit. After that everything is arbitrary. (Also technical limit is to a certain point arbitrary: int vs. bigint, text vs. mediumtext etc.) If you care for your product/platform, you end up limiting a lot of things or the moderators would quite possibly burn out very quickly...
    – ZZ-bb
    Commented Feb 4, 2016 at 14:15
79

This is exactly what closing a question as "Too Broad" tries to prevent. This question and your answer simply are not suitable for the Stack Overflow Q&A format.

As suggested, you could write a blog post (or multiple) and make your answer in list form and link to that blog, but that won't take away that the question is too broad. Its answers are a collection of "tips and tricks" that may or may not be true, and may or may not work.

Besides that, the definitive answer to "How can I prevent scraping of my website?", is missing, namely "Given a determined enough scraper, you can't".

1
  • 46
    Most likely that last line was beyond the 30K limit.
    – Jongware
    Commented Feb 2, 2016 at 18:27
10

The fact that you needed so many words to even attempt to answer the question should have been your first clue that maybe, just maybe, you shouldn't be trying to answer it at all.

Apart from the fact that this question is the very definition of the "too broad" close reason, it's also over half a decade old. Why you felt the need to answer an off-topic, ancient question is beyond me. Why you felt the need to answer such a question with over thirty thousand characters is... well, I thought I was unnecessarily verbose. :p

Answering an overly broad question with an overly broad answer is not helping. Sticking a bounty on the question to prevent it from being closed - as it should be - makes it seem that you're trying to keep the question open purely so your answer can get some screen time. (This is a harsh and perhaps unfair viewpoint, but I have relatively little information to go on, so please correct me if this is an invalid assumption on my part.)

As such, I've raised a moderator flag requesting the question to be closed. If that amounts to nothing, I will cast a close vote when the bounty(ies) expire.

Your answer, in its current form, is a great fit for a blog post or GitHub, but not for Stack Overflow, and not on that question.

5
  • 2
    As, I said, I've learned a lot about SO - and I won't post an answer like this again. I answered this question because the other answers where incomplete and outdated, and I felt I could contribute something good, something better than my usual FGITW answers. With the bounty - You're partially right, but the intention wasn't really to "get my answer some scree time" - and the intention most certainly wasn't to prevent closure. Rather I started the bounty after my answer sat there at the very end of the page for weeks, and it seemed nobody would ever see it and benefit from it. Commented Feb 3, 2016 at 9:26
  • 1
    As a note: The bounty has actually resulted in two other new, fairly high quality answers, so even though the question is off-topic, I think this is still a benefit. (oh, and finally: a moderator closing the question would require a refund of the bounty (?), which is probably not such a good thing (?)) Commented Feb 3, 2016 at 9:30
  • @JonasCz What you're saying is that you wrote your very long answer and nobody read it, so you put a bounty on the question to get people to read your answer. Regardless of the reason, that's still self-promotion - bounties are for askers, or people who find a question that they think is neglected... But I'm not going to harp on about this because I think you understand the site better now, and because you're actually contributing good content. In future, however, please ensure you reserve that good content for good questions! :)
    – Ian Kemp
    Commented Feb 3, 2016 at 11:31
  • 2
    @IanKemp I want to point something out: It has been discussed on Meta before, that using bounties to promote your own answer is not against the rules since 1) it could result in more, better answers by other users 2) you can't reward yourself anyway and 3) you have to pay double the minimum if you start a bounty on a question you already answered, so if you don't get 10 upvotes after posting the bounty, you're out rep anyway. (Or if you get downvotes rather than upvotes.) See also: meta.stackoverflow.com/q/286103
    – Kendra
    Commented Feb 3, 2016 at 15:40
  • As to the close vote - It got closed, 5 people thought it should he reopened, so it got reopened, but hey, now I was able to spend one of my first close votes after reaching 3k on it :-) Commented Feb 4, 2016 at 20:24
9

"my answer: It's a list of things"

You may have tried to cram multiple answers in a single answer. While not generally encouraged* , you are certainly allowed to post multiple answers, and certainly if each answer highlights the conditions in which it's appropriate.

*) People new to this site occasionally add an answer when trying to "reply" to another answer. That's bad enough, but if the answerer replies with yet another answer the Q&A format breaks down into unintended dialogue. Another reason is to prevent reposting an improved answer as a second answer, instead of just editing the original answer.

8

You can split this big answer into separate parts (an answer per solution) and refer to related answers by hyperlinks if it is nessesary.

The other way is to shrink the length by the formatting. Some examples:

  • If you have a lot of code formatted by Ctrl+K you are spending 4 chars per line, but if you just wrap the code into <pre><code>...</code></pre> you'll spent only 24 chars for the whole block.

  • Now you can format code even shorter - use ``` on separate lines at the beginning and ending - just 8 chars for the whole block.

  • In the code you can replace spaces by tabs.

  • You don't need code marking (4 spaces indentation or <pre><code>) inside of the snippets, but the system inserts the code with 4 spaces like the usual code.

  • In the answer you can use 2 spaces at the end of the line instead of <br>.

  • You can use inline hyperlinks without explicit protocol: [Google](//google.com): Google

  • For the links to other questions or answers you don't need to write the site and can drop the title: [This question](/questions/316012): This question.
    Or even shorter: [This question](/q/316012): This question.

  • You can skip repeating of the marker for the multiline quote inside one paragraph.

I'm sure a lot of formatting tricks can be applied.

4
  • The formatting tips are great, thanks ! That would also mean less typing. I disagree about spiltting it up though, as then it would not be one coherent piece anymore, the order would not be right (due to voting andtye default sort order), and other reasons I can't really think of right now. Commented Feb 3, 2016 at 13:51
  • 1
    @JonasCz, I ment, that if you have 4 ways to solve the problem, you can post each solution in separate answer, not just to split the answer into parts.
    – Qwertiy
    Commented Feb 3, 2016 at 13:53
  • I get you, the thing is that here, I've got 20 ways to solve the problem.. Commented Feb 3, 2016 at 13:54
  • @JonasCz, what about splitting into cattegories? Like "Ways acceptable if ... are, but for the other cases see <link to the other answer>." But yep, the can be cases where splitting is not a good solution.
    – Qwertiy
    Commented Feb 3, 2016 at 13:56
4

Perhaps this would fit better on Stack Overflow if you broke up the question into multiple, more specific questions, and then posted the specific parts of your answer to each of those questions.

It would solve the issue of the 30k limit while keeping all the content on Stack Overflow, and in a way that would benefit users who are only interested in some part of the answer. You could cross-link the questions/answers if it makes sense.

1
  • 3
    For any other (compound) question, this would be the proper solution. For that specific question: it would only be more questions to close as "waaay too broad".
    – Jongware
    Commented Feb 2, 2016 at 22:51
4

I reviewed the GitHub version of your answer. I would say you haven't made a serious effort to edit it for length.

  • The first four paragraphs are just blather and can be deleted without loss.
  • There is no functional difference between bullet points 2 and 3, only a difference in how they are constructed. (This also contradicts the prior claim that 'each works differently'.)
  • Don't put punctuation in headings, and don't put spaces before question marks, or around / when used to express alternation.
  • You don't need both the 'General' heading and the short paragraph that follows it.
  • Lots of blather, such as 'specifically, some ideas' where only the first word is necessary; another nearby example is 'it is a good idea to ...', which is entirely redundant. Ditto '[some] indicators ... [include]', where the words in brackets are redundant. Ditto 'in order to ... [, you should]:'.
  • You have at least two separate sections about Captcha. Combine them.

That's as far as I got, but it is clear there is considerable room for improvement. I would get someone else to look at it for you and blue-pencil it ruthlessly.

1
  • Thanks for your input ! In fact, I've already shortened the version here on Stack Overflow, and done most of the things you mentioned. In fact, I haven't edited the GitHub one for length at all yet., I agree that there's lots of room for improvement, andthat it's also a good idea for someone else to look at it. Commented Feb 4, 2016 at 7:36
3

This is something that just popped to my mind, but this kind of questions, ”How do I do X”, kind of feel more like requesting a tutorial, so this kind of questions more of belongs to, in my mind, to the documentation side.

I don’t know if the upcoming documentation SE actually allows this kind of vague, not project related topics, but quite often there is things that is not obvious, where information where the thing can be done scattered all around, incomplete and/or obsolete, so I see value in long answers explaining how the thing is supposed to be done, while maybe out of scope on Stack Overflow, maybe better suited to more documentation related site. I mean, Stack Overflow is meant to be used for simple, specific questions, rather than as a service that gives howtos about how a thing can be done, what needs to be taken in consideration and other generic usage notes.

The documentation site would contain generic instruction about how this thing should be used, and Stack Overflow would cover specific problems, like, why I couldn’t do the crawler in this way if I want to achieve this thing in order to produce that, which may be too unrelevant case to include in the main documentation.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .