10

What are all the possible calculations that could cause a NaN in Python?

The original version of this question, reproduced below, is asking what possible ways there are to get a NaN in Python. The current version asks the same about NumPy and SciPy. It was closed as too broad, then it got reopened, and now people are trying to close it as too broad again.

This question looks to me to be clear, narrow, and useful. Is this simply a case of people casting close votes on a subject they aren't sufficiently familiar with?

Original question:

Okay I've been searching around and there appear to be scattered discussions about NaNs in different programming languages, including some specific cases, but not exhaustive or clear.

I have one simple question:

What are all the different operations that would cause a NaN, in python?

Current question:

Okay I've been searching around and there appear to be scattered discussions about NaNs in different programming languages, including some specific cases, but not exhaustive or clear.

I have one simple question:

What are the most common operations that would cause a NaN, in python, which originate while working with numpy/scipy?

37
  • 16
    It's a list question. SO isn't designed to support list questions.
    – Servy
    Commented Aug 26, 2014 at 18:11
  • 3
    @Servy: It does not appear to be a "list question" according to Shog9's response at meta.stackexchange.com/questions/124450/… .
    – tmyklebu
    Commented Aug 26, 2014 at 18:18
  • 7
    I think it's a legit question. (before and after edit) If it were put better, it would help future readers decently, and plus, it doesn't attract low-quality answers.
    – Unihedron
    Commented Aug 26, 2014 at 18:21
  • 8
    @Unihedron There's plenty of useful information that simply isn't well suited to the format of SO. Just because some information is useful doesn't mean it has to be able to be an SO question.
    – Servy
    Commented Aug 26, 2014 at 18:23
  • 8
    @Servy And how is this question asking for useful information that isn't well-suited as a SO question? It doesn't match the Too Broad qualification: "There are either too many possible answers" (not true as there are a finite and documented operations)", or good answers would be too long for this format." (not true as the question requests the information with the specification "working with numpy/scipy") The question doesn't have to "add details to narrow the answer set or to isolate an issue that can be answered in a few paragraphs." either.
    – Unihedron
    Commented Aug 26, 2014 at 18:28
  • 8
    @Unihedron Before the edits it absolutely was. When looking at the set of "all python code" then any custom function can potentially return NAN, for whatever reasons the implementer of that method feels it should (whether sensible or not). It is an infinite set of always increasing items. Limiting it's scope to "common" functions, after the edit, does decrease the set of items, but also adds in a strong subjective factor. What's "common" to one person is not common to someone else. Limiting the scope to just a specific set of libraries (the edit) also does help, but only to a point.
    – Servy
    Commented Aug 26, 2014 at 18:34
  • 10
    @tmyklebu I never said there was anything subjective about whether or not an operation returns NaN. I said it was subjective wither or not the function is "common" or "uncommon". Oh, and just because you only listed 5 operations doesn't mean the set of valid operations isn't infinite, it only means that you haven't listed anywhere near the full set of operations that can return NaN. You appear to either not know what floating point numbers are or you're deliberately misinterpreting my comments in order to cast the question as not "too broad".
    – Servy
    Commented Aug 26, 2014 at 18:39
  • 3
    @tmyklebu Except that it's not.
    – Servy
    Commented Aug 26, 2014 at 18:41
  • 8
    @tmyklebu Computing the exponent of a value, Sin, Cos, Tan, negation, just to name a few off of the top of my head. I'd expect that the vast majority of all operations you could perform on a floating point value could return NaN (certainly just about any sensible implementatino of any operation should have that capacity, although some unsensible implementations or unusual operations won't), including whatever operations you write yourself, which is of course what makes the set infinite.
    – Servy
    Commented Aug 26, 2014 at 18:59
  • 3
    @Servy: Further, even if you were advancing a sane argument, I don't think we should close this question simply because the asker didn't know to say "floating-point operation defined by the IEEE 754 standard" or whatever. If he knew how to say precisely what he meant, he'd know where (the IEEE 754 standard) to look for the answer.
    – tmyklebu
    Commented Aug 26, 2014 at 19:11
  • 6
    Given that your answer punts on the idea of providing a full list in two different places, it's hard to see how you don't see how other people could think this is too broad.
    – jscs
    Commented Aug 26, 2014 at 19:54
  • 7
    @tmyklebu If you limit the scope of the question to just IEEE defined operations then the question has a small finite set of operations to evaluate and answer on. Without it it has an infinite set of items and your answer is radically incomplete. If you think limiting the scope of the question to just IEEE operations is limiting it's usefulness then that's admitting that the current scope of the question is not just to IEEE operations, that it is asking about the infinite set of all possible floating point operations, and that your answer is very incomplete.
    – Servy
    Commented Aug 26, 2014 at 20:06
  • 7
    It's punting on the idea of providing the exhaustive list that the OP is supposedly asking for: "These five things, plus -- potentially -- a bunch of other things in this category (you'll have to look those up for yourself)."
    – jscs
    Commented Aug 26, 2014 at 20:10
  • 3
    @tmyklebu Then write the question so that they don't need to understand that term before comprehending the question. That's easy enough to do. And again, when you aren't binding the set of items to just those defined by IEEE the set is infinite, and the question currently does not do this. Currently you're pretending that the question is a dramatically different question than it currently reads, and you're trying to answer that imagined question, rather than what is there.
    – Servy
    Commented Aug 26, 2014 at 20:10
  • 8
    Any "how do I" question is, based on what I read from this discussion, too broad in the world of programming, because there are a potentially infinite number of ways to do just about anything. Perhaps we should just suck it up and allow SO to make its transition from a knowledgebase into a debugging site. That way nobody will have to worry about broadness of anything anymore.
    – BoltClock Mod
    Commented Aug 27, 2014 at 16:39

5 Answers 5

10

I disagree that the question is Too Broad (post-edit). I think it's a legit question; If it were put better, it would help future readers decently.

The Too Broad qualification is:

"There are either too many possible answers, or good answers would be too long for this format."

  • The question doesn't have too many possible answers as there are a finite and documented amount of operations, and the scope and expression is clear -

    calculations that could cause a NaN

  • The question doesn't require an answer too long to answer either, as the question requests the information with the specification "working with numpy/scipy".
  • The question doesn't have to "add details to narrow the answer set or to isolate an issue that can be answered in a few paragraphs." either.

Also, low-quality questions (that one should rush to close) tend to attract low quality answers. This doesn't.

While it may be subjective, it does not generate discussion, and opinion-based posts as the qualification of a question that is primarily opinion-based.

I disagree that the post is Too Broad, less to deserve being closed for the second time (post-edit).

18
  • 13
    At the risk of offending those who follow the rules to the word, I agree with this. Even if it is asking for a list, I see no harm if the list is short. There's only a few ways you can produce a NaN.
    – Mysticial
    Commented Aug 26, 2014 at 18:43
  • 4
    @Mysticial: Is there even a rule against this question? All I've been able to get out of Servy is basically "I don't like questions that I don't understand."
    – tmyklebu
    Commented Aug 26, 2014 at 18:53
  • 4
    @tmyklebu Even then they're using the wrong close reason...
    – Unihedron
    Commented Aug 26, 2014 at 18:55
  • 11
    @tmyklebu Welcome to meta.SO. The mecca of pointless bickering over 1st world problems.
    – Mysticial
    Commented Aug 26, 2014 at 18:55
  • 1
    @Mysticial: I gather I shouldn't actually be trying to follow seemingly-insane advice dispensed here?
    – tmyklebu
    Commented Aug 26, 2014 at 18:56
  • 10
    "finite and documented amount of operations" - there's a finite, small, and documented number of operators, but operations? There is a crazy number of functions and classes and weird interactions that might produce NaNs, and more are invented constantly. Commented Aug 26, 2014 at 23:30
  • 2
    @user2357112: I debunked that already. Go read the spec.
    – tmyklebu
    Commented Aug 27, 2014 at 8:54
  • @tmyklebu: You have an extremely restricted interpretation of "operation" that the actual question does not support. Commented Aug 27, 2014 at 8:59
  • 1
    @user2357112: I debunked that already. Go read the spec. The spec is the IEEE 754 standard. Look how it uses the word "operation."
    – tmyklebu
    Commented Aug 27, 2014 at 9:00
  • 1
    @user2357112: And, even if the question were "too broad" by the book, I can't support closing questions like this. It's an "if you knew how to phrase your question precisely, then you'd know where to look for the answer" question. Closing these for reasons other than "duplicate" does nobody any favours.
    – tmyklebu
    Commented Aug 27, 2014 at 9:03
  • 4
    @tmyklebu: What makes you think the questioner is referring to the IEEE 754 standard's use of the term "operation"? It's a NumPy/SciPy question; why not use the term as the NumPy and SciPy docs do? You will find that that documentation uses the term in a much broader way than what you have in mind. Commented Aug 27, 2014 at 9:07
  • 2
    @tmyklebu: What gave you the idea the question was about machine arithmetic? There is not a single reference to machine arithmetic in the title, body, or tags, in any of the question's four revisions. Somehow, you've gotten the idea that this question is something it's not. It's making you miss things like the exponentiation operator (**), which is a common operation that produces NaNs, because you're thinking in a model where that's not an operator. Commented Aug 27, 2014 at 10:00
  • 2
    @tmyklebu: How else? Well, by using NumPy. I don't know why you'd look at a Python question, think "Python likes to turn NaNs into exceptions", and jump to the conclusion "The questioner must be asking about machine-level floating-point operations" rather than "The questioner is using libraries that don't do that with NaNs". Commented Aug 27, 2014 at 10:31
  • 1
    @tmyklebu "[...] when he uses NumPy to do tons of machine arithmetic" is an unjustified assumption on your part. I'd even guess it's a false assumption; read the comment by flebool and OPs response: "causes of NAN [...] that do not originate from other libraries [than numpy/scipy]" - which I would interpret as OP wanting to know all numpy operations and library functions which could ever return a NAN; I think you agree that question is too broad. FWIW, I'd close the question as unclear until OP clarifies if he wants to know about IEEE754 operations, or about all of python and numpy.
    – l4mpi
    Commented Aug 27, 2014 at 11:19
  • 3
    @tmyklebu and the comment that flebool just left in reply to you on the original question pretty much confirms my view: "Many libraries return (or can return) nan under certain circumstances, defined by the library, instead of returning an exception for example. They are a finite but very large number of cases.". Do you still think anybody who might find this question somewhat broad doesn't know what floating-point numbers are or deliberately misinterprets the question?
    – l4mpi
    Commented Aug 27, 2014 at 11:31
16

My problem with questions like this is not that they are too broad, but that they are already completely answered by the relevant specifications, in this case IEEE754.

I don't want SO to become a repository of transcriptions of those standards, or, worse, incorrect renditions of what they say, but both are inevitable if questions like this are permitted. The goal of SO is to become the first place you should look for answers to on-topic questions. As the first place you should look in such cases is certainly the relevant specification, that makes the question off-topic by definition.

13
  • 4
    why don't point to the specification as an answer ? I'm really asking Commented Aug 27, 2014 at 0:07
  • 4
    @Bart I agree, that's exactly what should be done, as opposed to posting extracts, even verbatim, and certainly as opposed to posting mangled paraphrases.
    – user207421
    Commented Aug 27, 2014 at 0:50
  • 5
    This is a good point, and I almost agree with it. The gap I see is that (0) specs can be expensive, (1) people don't always know which specs are relevant and which aren't (the use of NumPy here means you're getting native arithmetic, not Python arithmetic), and (2) specs aren't always easy to interpret. The IEEE spec is reasonably clear---section 7 of the IEEE spec talks about exceptions and how they're handled---and I could have done better to either transcribe the relevant part more accurately or give a pointer to the spec itself.
    – tmyklebu
    Commented Aug 27, 2014 at 7:41
  • @Bart: Because I didn't think of that. (Why didn't I think of that? Because I'm apparently daft.)
    – tmyklebu
    Commented Aug 27, 2014 at 8:02
  • 12
    Your point seems to be, if there is anything anywhere that would answer the question, it should be off-topic. I think otherwise, the StackOverflow should be the first place to look for an answer, with exception to language basics and API specifications. Commented Aug 27, 2014 at 8:29
  • 5
    Aren't most (or at least many) questions on SO solvable with the help of the specification? I've seen a ton of questions a la "Why does operation X(YZ) not work", while reading the documentation reveals that the required parameter would have been ΨΩ. Edit: I agree with @Donaudampfschifffreizeitfahrt
    – Theolodis
    Commented Aug 27, 2014 at 8:33
  • 2
    @Theolodis: Yes, but whether it's reasonable to expect the asker to understand the specification varies with the specification. IEEE 754 is pretty simple---I probably could have answered with a link and a few pointers about interpretation. C++, for example, is not simple---have a look at the language-lawyer tag.
    – tmyklebu
    Commented Aug 27, 2014 at 8:43
  • 2
    @tmyklebu I do not know the IEEE 754 (c++ programmer), and I am sure not many people actually do. But that makes an answer on SO, with a link to the specification!!!!, even more useful ;)
    – Theolodis
    Commented Aug 27, 2014 at 8:48
  • @Donaudampfschifffreizeitfahrt That is a complete and utter misrepresentation of what I said, for which there is absolutely no excuse. My point is that when a normative reference or specification is applicable (as per your own final sentence, which isn't too different from what I wrote), it is otiose to quote it, and even worse to paraphrase it.
    – user207421
    Commented Aug 27, 2014 at 9:58
  • 2
    @EJP: I dunno about "otiose"; as I said, some specs cost money, others are hard to follow, and it may not be obvious which specs are applicable in the first place.
    – tmyklebu
    Commented Aug 27, 2014 at 10:18
  • @tmyklebu Agreed but we are discussing 'questions like this', not abstruse things buried deep in specifications that are only comprehensible to implementors.
    – user207421
    Commented Aug 27, 2014 at 10:27
  • 1
    @EJP: Sure. But that means the rule should be "if there's a spec, provide an interpretation and for the love of god also a reference" or thereabouts, rather than "if there's a spec, don't repeat it." My initial failure to do so was an oversight on my part.
    – tmyklebu
    Commented Aug 27, 2014 at 10:37
  • I understand that not all users are willing to take the time to explain a spec in plain English in their own words, but in general I disagree that everyone posting on SO should have to study the spec for everything before asking; that seems like overkill especially since some of these specs are behind a paywall, and/or bloated with outdated/esoteric information, "please don't sue me if it doesn't work" legalese, or terminology that only a compiler writer/hardware designer would know or need to know about.
    – jrh
    Commented Oct 18, 2017 at 17:47
12

The original question was (and the title still is) too broad. It's like asking for a complete list of functions that can raise ValueError.

The question limited to NumPy / SciPy may be too broad, but is more likely to get good answers, even if the list is never comprehensive.

8

I don't find this question "clear": it asks about "common operations in Python" - no one knows what that is, and even if you claim that you do - you don't, unless you show me some convincing statistical data from a significantly large number of Python programs written.

I don't find this question "narrow": as the number combinations that leads to this outcome is infinitely larger then the list in the answer. Even if the answer is good and narrow, doesn't mean the question is. The answer deals with "operation" in IEEE's meaning - not referring to what an "operation" is in Python. Here is an example of a Python operation not listed in your answer:

class X:
    def __add__(self, other):
        return 1/0

print X() + 1  # X() + 1 is an operation

I don't find this question "useful": why would someone need a list like this? Don't they teach people in the kindergarten not to divide by 0? Or isn't it obvious that something's wrong with your code when it tries to come up with an answer to "how much is it inf - inf"? Anyway, even if someone will not understand why it's wrong, they can post their code and ask what's wrong with it - that would be narrow.

The conclusion is that IMHO this should be indeed closed as too broad, or migrated to Programmers. The latter seems more appropriate because the answer is indeed very nice.

Another possibility is to ask a different, better and more precise question, and self-answer it.

8
  • 1
    What's wrong with dividing by zero?
    – tmyklebu
    Commented Aug 27, 2014 at 8:50
  • And your conclusion is?...
    – Unihedron
    Commented Aug 27, 2014 at 8:55
  • Separately, your "not clear" complaint is a red herring, your "not narrow" complaint has already been debunked, and the question's useful because it pays to understand the behaviour of the things you're working with.
    – tmyklebu
    Commented Aug 27, 2014 at 8:57
  • @tmyklebu 1) It's a direct response to your diagnosis, so if this is a red herring then so your "the question is clear" is. 2) No it wasn't. The question is about Python, in which an operation is something very different than IEEE's operation. 3) It doesn't help to understand "things you're working with" because there is no thing someone was working with in the question. Your (albeit very nice) answer won't stop people from posting a particular issue that results in such errors, even if they will read it.
    – BartoszKP
    Commented Aug 27, 2014 at 9:06
  • ... and that's because the problem mostly is not that people don't know that you shouldn't divide by zero (except you, apparently), but they often don't know how come during execution of their code it comes to the point when division by 0 occurs.
    – BartoszKP
    Commented Aug 27, 2014 at 9:12
  • @BartoszKP: Nothing will stop people from posting about a particular issue in which a NaN arises, nor should it. "Scientific" code is usually written with the "useful tool that usually works" goal in mind rather than the "bulletproof library that satisfies all of your naive wishes" goal. This is because the "bulletproof" variants wind up being unusably slow, often requiring arithmetic on huge numbers. By its very nature, then, this stuff is considerably less composable than "integer" code. What follows is a need to understand the implementation at at least some level.
    – tmyklebu
    Commented Aug 27, 2014 at 9:50
  • @BartoszKP: Perhaps "operation" means something different in Python. I don't know---I'm not much of a Python programmer. If so, the correct course of action is to correct the poster's clumsy wording rather than close the question.
    – tmyklebu
    Commented Aug 27, 2014 at 9:52
  • 2
    @tmyklebu If you are really sure what did the OP mean. That would however be a gray area edit. After all, the Python tag most probably isn't there by accident. This suggests again that you really wanted to answer a question you had in mind, and just found something closest to it. Perhaps consider asking a better question yourself, and self-answering it? That would be the cleanest solution according to SO policy IMHO.
    – BartoszKP
    Commented Aug 27, 2014 at 9:55
-4

The question, as originally asked, was absolutely too broad: the number of Python "calculations" that can produce a NaN is essentially unbounded, limited only by the combined creativity of the Python programming community. Replacing "all the possible" with "the most common" just makes it subjective, too; arguably, it's still too broad, as there's a limitless number of possible answers depending on each answerer's definition of "most common".

That said, the question has a nice answer, which I'd hate to see lost. So what we should do is figure out what question the answer actually answers, and then edit the question to actually ask it.

The trick here, I believe, is to be specific. The help center suggests that a good SO question should be about "a specific programming problem". While the original question was written in the form of a (doomed) literature search looking for an exhaustive list of possible sources of NaNs in Python, the specific programming problem behind it, which the answer correctly picks out, seems to be something like:

Where do NaNs come from?

I wrote some Python code (using numpy / scipy) to calculate a numerical value, but it returned NaN instead. It seems that any further calculations I do with this NaN value also return NaN, but I'm sure that my original inputs didn't have any NaN values in them.

What are these NaN values, what kinds of operations produce them, and how can I find their source and fix it?

(This is just a quick draft; feel free to suggest improvements.)

IMO, rephrased like that, this is a valid and even a good question for SO: not so broad that it would be unanswerable, nor so narrow that it wouldn't be of use to other people. True, the lack of program code means we can't debug it ourselves and find the specific source of the OP's NaN, but, as the existing answer demonstrates, that doesn't necessarily preclude us from providing a general answer that describes the typical sources of NaNs and how to locate them.

Indeed, it could be argued that, for future visitors finding this question and answer on Google, such a general answer can actually be more useful than one that focuses on finding and fixing a particular bug in a particular piece of code.

Anyway, unless someone beats me to it (or unless this answer gets downvoted to oblivion), I'm going to edit the question later today to say something like my suggested wording above.

2
  • I think your proposed rewrite makes it even broader (asking four related but separately answerable questions). "How do I debug" questions are generally not on-topic, and just saying "I wrote some code" is not preferred - we like minimal examples that reproduce the issue.
    – nobody
    Commented Aug 27, 2014 at 17:43
  • 1
    You have my blessing, for whatever it's worth, to edit the question. But I still don't see why people are making a fuss over the wording here and I'm perfectly happy to let the question fade into oblivion if the community prefers that.
    – tmyklebu
    Commented Aug 27, 2014 at 20:13

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .