29

In this question, OP reports being unable to compile a very large (about 20 MiB) C source file, gcc just doesn't terminate. This problem is most likely caused by the source file being very large. For that reason, the source file cannot be reduced to a more reasonable size, i. e. small enough to be posted inline in the question's body. OP uploaded the source file to a file-sharing site, which earned him downvotes and (almost) a closed question.

What is the recommended process for askers in such a situation?

12
  • 2
    @HovercraftFullOfEels OP said he generated this file, I assume it's the “natural” representation of some formula. Anyway, for this question it would like you to take for granted that the problem cannot be reproduced in a smaller source code form.
    – fuz
    Oct 31, 2015 at 20:49
  • 1
    Sorry, I did not see that information in your question above. Oct 31, 2015 at 20:50
  • 35
    It is rather trivial to post the code to generate a 20 MB source code file. A few lines of code gets that job done. And a trivial way to check whether it is actually the size or the file that matters. If it is in fact the content and not the size, well, then of course it is up to the OP to use the delete key in his editor before posting. Oct 31, 2015 at 22:48
  • I'd suggest the OP to provide an MCVE. Nov 2, 2015 at 11:02
  • 3
    @FrankN.Stein The prerequisite to this question is that that cannot be done.
    – fuz
    Nov 2, 2015 at 11:24
  • @FUZxxl Then I suppose that the OP is helpless. Nobody is going to download such a big file and analyze it for the OP. Unless the OP pays big money. Nov 2, 2015 at 11:26
  • 4
    @FrankN.Stein Well, he received an answer (namely, use a different compiler until the gcc folks fix their compiler bugs).
    – fuz
    Nov 2, 2015 at 11:38
  • @FUZxxl Such an answer can be given without even seeing a single line of code ;) Nov 2, 2015 at 11:39
  • 4
    @FrankN.Stein No. I only found this answer after fiddling around with OP's file, trying different compiler options and compilers.
    – fuz
    Nov 2, 2015 at 11:40
  • @FUZxxl (!!) Lucky you, who have a lot of time to spend. ;) Nov 2, 2015 at 11:42
  • 14
    @FrankN.Stein People charitably spending time on other people's problems in order to win fake internet points is...90% of the reason this site is effective. Don't try to invalidate the question because it doesn't fit your preconceptions of what is or is not answerable. Nov 2, 2015 at 19:13
  • @Hans: trivial to post the original code, perhaps, but nobody wanted to spend $2000 for the product that could compile it. :-) Nov 3, 2015 at 20:26

4 Answers 4

46

There is no built-in way to provide a 20MB file within Stack Overflow.

An external link is the only way to go.

Luckily, questions where linking to such a file is required and justifiable are extremely rare. They should stay that way, as they're not a class of very useful questions in the long run. If the external link breaks, the Q&A will not be of much use to future generations.

6
  • 6
    "questions where linking to such a file is required and justifiable are extremely rare" -- I've yet to see even one. So far, there doesn't seem to be any evidence that the one that prompted the OP's question here is an example of such. Nov 1, 2015 at 23:13
  • 9
    I disagree with the last statement "If the external link breaks, the Q&A will not be of much use to future generations." It depends entirely on rest of the question and the answer. In that case, it is not true. All the visitor has to do is give clang a try and maybe it will work (on their code, not the OP's). It is guaranteed that they won't be generating the exact same 20 MB source file as that OP, so the code is already useless to everyone now that the question was answered. Besides, the greatest value is to help the OP, in my opinion. Maybe his MATLAB project will cure cancer. Who knows. Nov 2, 2015 at 1:24
  • _They should stay that way, as they're not a class of very useful questions in the long run. _ Did you read the question? It seems like it actually may be important for others as compiler limits (especially when dealing with generated source) are unreasonable to crop up multiple times for multiple people.
    – Sled
    Nov 2, 2015 at 17:01
  • @user1122069 Sure, and maybe the person who is working through their program line by line posting questions which could easily be answered by paying attention to a basic manual or tutorial is also working on a project to cure cancer. That is not, however, a reason for considering the question itself either "on topic" or "high quality" - there is no consideration of motive in the Help Center, and it is the question itself that needs to be useful in future, not the impact it had on somebody's life. That's different from, say, a forum, where "making people happy" might be an explicit aim.
    – IMSoP
    Nov 2, 2015 at 18:28
  • @ArtB it's still a good thing that questions that come with a 20MB file are rare.
    – Pekka
    Nov 2, 2015 at 19:11
  • 1
    @Pekka웃 agreed, but this shows that they can be legitimate and useful... something I doubted until I read the question itself. I only suggest everyone on this thread(?) take time to read the question itself to see that it does belong before having a knee-jerk reaction (like I initially did).
    – Sled
    Nov 2, 2015 at 22:19
33

Not every single programming problem must be posted as a Stack Overflow question.

We are very clear that debugging requests must be accompanied by an MCVE. If that is not feasible due to the nature of the problem then, while it is unfortunate for the OP, the rules do not magically change. This is not a blind adherence to those rules: indeed, it is a perfect example of why we have them in the first place. This is not a helpdesk. Questions must remain useful in the future. A question dependent on an off-site link is, by definition, not so.

The OP could instead find a more transient form of discourse in order to receive help with their problem. Perhaps one of our many chat rooms, accompanied by an off-site link to the problem proof? Or, like, talking to a person in real life maybe.

24
  • 6
    So you say that problems that cannot be reduced to a sufficiently small size are automatically off-topic? I think that a question of the form “gcc hangs when compiling a very large source file” are both on-topic and highly useful for future readers.
    – fuz
    Nov 1, 2015 at 19:48
  • 2
    @FUZxxl: I agree, as long as there is sufficient information in the question to answer it. The same logic applies here as it does to any such question. My key point is that making exceptions, like "don't worry this time, just post an MCVE for your next question", is fundamentally the wrong approach. We don't have to spend any effort trying to shoehorn every single permutation of programming problem onto SO — sometimes, this just isn't the place. And that's okay! There are other places. Nov 1, 2015 at 19:51
  • 4
    Could this question reasonably be answered without having access to the megafile? Likely not. The OP does not say anything at all about its contents, all inferences on its contents are from others, and the OP was looking originally for something to change in his gcc. Technically, that makes it "not reproducable". "A learning experience is one of those things that says, 'You know that thing you just did? Don't do that.'" (Douglas Adams)
    – Jongware
    Nov 1, 2015 at 21:55
  • 2
    @FUZxxl: "problems that cannot be reduced to a sufficiently small size are automatically off-topic?" -- Have you seen such a question? Because the one you've referred to above doesn't appear to be one. You yourself stated that the code was generated, and as Hans pointed out, the code to generate the code likely is well within normal Stack Overflow limits. Conversely, a large source file that cannot be generated but which reproduces a bug most likely could be distilled into something much smaller that still reproduces the same bug. Either way, the whole 20MB isn't actually required. Nov 1, 2015 at 23:12
  • 3
    @PeterDuniho: Right, but it wasn't distilled. Until that occurs, the question is not appropriate here. Nov 2, 2015 at 0:22
  • 1
    @LightnessRacesinOrbit: agreed...that's why in my comment to the other answer, I wrote "I've yet to see even one". Nothing in my comment here should be construed as an endorsement of the question in question. I'm simply commenting on the theoretical scenario commenter FUZxxl postulates. (All that said, the question in question does seem to have useful answers, and as questions go it's far from the worst I've ever seen). Nov 2, 2015 at 0:26
  • What? I've seen a MCVE that was 20 GIGS before. (Problem was caused by auto-rebuild-statistics that simply would not show itself on a smaller dataset.)
    – Joshua
    Nov 2, 2015 at 21:23
  • 1
    @Joshua: What what? I'm not saying it never happens. I'm just saying that when it does happen (which I imagine is exceedingly rare, not including all the millions of times per day an OP could and should have constructed a 15 line MCVE but didn't...) the question is not suitable for the Stack Exchange format. And that it's a big wide world out there and the programmer shouldn't be afraid of finding alternative venues for their free help. Nov 2, 2015 at 21:44
  • @PeterDuniho: the problem being that the source code in question was for MATLAB, and the people trying to answer the question didn't want to spend $2000 in order to do so. I really think that an off-site link, for the benefit of those attempting to reproduce the problem, was entirely appropriate in this case. Zero tolerance policies are not a good thing. Nov 3, 2015 at 20:24
  • 1
    @HarryJohnston: You have misunderstood what this website is for. "for the benefit of those attempting to reproduce the problem" No, the problem must already be reproducible. This is not a forum, or a chat room, or a crowdsourced debugging tool. Simple as that. Nov 3, 2015 at 23:49
  • @LightnessRacesinOrbit: but the problem was reproducible, and it wasn't a debugging request. The question and answer are useful to future visitors as-is and meet all of the other requirements for SO. So your objection seems to be based entirely on the fact that, in order to figure out the answer, it was necessary for people to download a large file. I don't think that makes sense. Nov 4, 2015 at 0:03
  • (In fact, it wasn't even necessary. The availability of the file just meant that it could be answered by someone who didn't already own MATLAB.) Nov 4, 2015 at 0:05
  • 1
    Reproducible using steps given in the question. Again, you're viewing Stack Overflow as a place where you can recruit debuggers for any transient problem. That's not the case. It's a database, of knowledge. If the question does not in and of itself constitute usefulness for future visitors then by definition it is off-topic. The fact that this problem could not be reproduced without downloading some massive file from somewhere else on the internet is a textbook example of that, I'm afraid. (cont.) Nov 4, 2015 at 0:26
  • 1
    (cont.) People say "it should be a help site", but it's not. Of course, the "it's not" is really an abstraction used to ensure quality. That's what sets this network apart from others, and makes it so vastly successful. It's important we don't stray from that. Nov 4, 2015 at 0:28
  • But it wasn't a transient problem, didn't ask for debuggers, and is as likely to be useful for future visitors as any typical question. Unless you're arguing that problems related to MATLAB are off-topic? And as I've already pointed out it could have been reproduced without downloading a massive file - the OP could have posted the MATLAB code instead. It's just that the particular people who chose to investigate the problem told the OP that the C file would be more useful to them. So it seems to me that everything you're basing your conclusion on is false. Nov 4, 2015 at 19:57
20

What the OP should have done was write a small program that generated a 20MB source file that also failed to compile (and, incidentally, did not require a proprietary header). The problem was not that that particular 20MB C source file did not compile (if that was the issue, it would not have been a helpful question to hang around), but that very large C source files of a given type (mostly consisting of a single expression) don't compile. Whilst posting an MCVE would have been hard, posting a program that generated an MCVE would have been relatively easy, and would probably have generated a far less negative response. It's also what the gcc developers would desire in order to address the problem.

4
  • 2
    I have no idea why Hans Passant didn't post this himself, given his comment. +1. Nov 2, 2015 at 19:17
  • I agree with this. The OP should have shown the program that generated that ridiculous expression. It was an XY-problem after all. That said, I just tried to write a Python script to reproduce the observed problem – and I failed. Generated a single 30 MiB expression and GCC compiled it in less than a minute on -O0. On -O3 I hit my 8 GiB RAM before the compile was done, though. But this contradicts what the OP was seeing.
    – 5gon12eder
    Nov 2, 2015 at 21:31
  • 1
    @5gon12eder I suspect he could not have posted the program that generated it as it was closed source. But he could equally have posted a program that generated a similar one line 20MB definition of an array of doubles.
    – abligh
    Nov 2, 2015 at 21:51
  • Since the problem was (probably) related to the structure of the expression, attempting to generate such a file from first principles would likely be a needle-in-a-haystack exercise. Nov 3, 2015 at 20:21
-23

As some have recently reminded me, down votes on a question are separate from close votes. While close votes require a reason, down votes can be entirely arbitrary. If it didn't actually get closed, that means that no official action was taken. Therefore, there was nothing "officially" determined to be wrong. Its all then just opinion. Someone won the election to the presidency, but that does not directly mean anything about his character, whether it is good or bad.

Well written, perfectly justified posts get down voted all day long. Nothing is "officially" wrong with that, however cruel and offensive.

It is an open/democratic system. The benefit of that is if your post doesn't get closed, it can still be answered. And even though down-voted and flagged for close (rejected) the OP will get a positive rating change overall from your up-voted and accepted answer.

Your answer there is 100% relevant even without the external link. So, who said there is anything strictly wrong? If they did, it is opinion.

Personally, I find it offensive, but if you want an answer, you can learn to live with it.

Take a look at the moderator's post again, and it is clear the external link was not the issue, but a quality question.

OK, I buy that the size of the file is the inherent issue here, and that others might have a similar issue, so I've reopened this. However, I am concerned that the linked file will go away at some point in the future, rendering the question less useful. That's why we tend to recommend people minimize the issue and place it within the question itself. This is a bit of an edge case. All I ask is that everyone keep their comments civil and on topic.

3
  • 6
    Man oh man. You should consider if SO is a healthy place to be in your case. It isn't going to change to your standards.
    – Gimby
    Nov 2, 2015 at 9:50
  • 1
    Stack Exchange sites are run by the community, not the moderators. Our version of “official” is community consensus. If the community downvotes your question to oblivion, the official stance is that it was bad. Nov 2, 2015 at 21:46
  • 1
    Besides, you don't need to be a ♦moderator to cast closevotes. Nov 2, 2015 at 21:50

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .