46

Questions like: https://stackoverflow.com/questions/55015042/even-if-i-change-the-hyperparameters-of-random-forest-the-test-score-wont-diffe attract primarily opinion-based responses, since there is no data available and the code to perform the task is correct.

However, putting such questions on Cross Validated might not be the solution, as they mostly deal with the theory of machine learning and not with debugging/tuning a model. As Stack Overflow expects questions that don't have primarily opinion-based answers, where do such questions actually fit? If they do fit on Stack Overflow, what should be the standard to answer them?

7
  • 4
    If there is no data available an MCVE is lacking in such a case, I think. Hence the question should be closed as off-topic.
    – Luuklag
    Mar 6, 2019 at 8:09
  • 1
    I was about to suggest to tell them to post at ai.stackexchange.com, but apparently implementation questions are explicitly off-topic there.
    – Lundin
    Mar 6, 2019 at 8:30
  • 2
    @Luuklag Closing the question as off-topic does seem to be the most adequate as per the current norm and guidelines, but then that raises the question what should someone facing such a problem do? In most cases someone cannot make their company's datasets public, should there be a community dedicated to tuning the models based on various plots of the data rather than the data itself? Mar 6, 2019 at 8:38
  • 7
    Well if someone is really interested in asking a question here on SO they should adhere to the guidelines. In such a case I'd say go the extra mile by building and providing a dummy dataset.
    – Luuklag
    Mar 6, 2019 at 8:47
  • 1
    Also on Cross Validated they maintain a list of other places to get help: stats.meta.stackexchange.com/questions/793/… quite a few machine learning options listed their for various programmes.
    – Luuklag
    Mar 6, 2019 at 9:10
  • 2
    There is also DataScience.SE, which welcomes applied machine learning questions.
    – hbaderts
    Mar 6, 2019 at 9:14
  • 11
    I believe "We have this off-topic question but I don't think it would fit well anywhere else" only implies "we have an off-topic question". I often face the fallacy that in this case off-topic is "fine" or "acceptable" (mostly from people posting off-topic content, of course). This is blatantly wrong. As far as we are concerned it's either a well-formed question as-per How to Ask or it isn't. Whether it can be asked elsewhere is a marginal concern that goes in comments under the closed question. And of course "don't migrate crap" is the golden rule. Mar 6, 2019 at 10:33

1 Answer 1

29

This seems to heavily relate to another Meta question. In my answer, I felt inclined to also discuss the on-topicness of machine learning questions, which I felt was the elephant in the room back then. Rather than repeating myself here, I will attempt to fetch some of its parts and work them here.

Regarding the question linked above, it seems to sit under the "how do I improve this model?" kind of open question. There may be cases where the obtained outputs are too different from the expected outcomes, suggesting that there is some kind of programming bug such as a misuse/misunderstanding of an API. However, this doesn't seem to be the case. Quoting from my other answer:

Questions seeking advice on "How do I improve this model?" are usually open ended, and will often attract generic recommendations that involve much more concepts than those in the scope of the site. They should be closed as Too Broad until they are severely narrowed down to a concrete concern. Providing advice here is tricky unless you are confident in proposing migration.

Without the data set, opinions are even more likely to flourish. The second major concern is indeed about making a Minimal, Complete and Verifiable example, which is more difficult here than one might think. This question not only lacks the data set, I also failed to identify the dimensionality of the features and labels from it, making the outputs even harder to reason about.

Hence, despite the subjectivity of some situations, the closest I'd get to an actionable rule set of machine learning questions would be: (1) identifying whether the question is purely theoretical; and (2) carefully analyzing the question to ensure that there is a proper MCVE.

1.

If you find that the question is purely theoretical, vote to close as off-topic and suggest visiting one of the sister sites. Do not vote to migrate unless you are absolutely sure that the question is in proper condition for that. I have occasionally employed a comment such as this one:

I'm voting to close this question as off-topic because it's purely specific to machine learning rather than programming. You may visit Cross Validated or Data Science SE instead.

This will give the OP an opportunity to have a look at the sites and understand how they function before pressing the Ask a Question button. While Cross Validated does not accept programming questions, the user might be able to adapt the question to a specific technology-agnostic issue. Data Science SE is more permissive than the other sites, likely as a consequence of its wide scope and low traffic. This one has generally accepted machine learning questions with code, as well as those where models are described in some other way. I am not implying that "anything goes". Naturally, it is up to the community at Data Science SE to set that bar in accordance to their expectations, and it's best not to make any assumptions about that when handling questions on SO.

Nowadays, it's also been commonplace to close these questions as not about programming since the merge of community specific closure reasons for Server Fault and Super User into one. That is fine too, assuming that the question is based on theoretical aspects of machine learning which deviate a lot from programming.

2.

If the question effectively narrows down to a programming problem, you may then put this matter aside and use the remaining criteria on whether the question is suitable for this site, with the MCVE being the most important. Things to look out for are, among others, the code for describing the model used, the solver algorithm for training the model, and the data set. Again, a few more details can be found in my other answer.

Other than that, one may also keep in mind that not all questions will fit our SE-specific paradigm of Q&A. Sure, there may be several legitimate questions being closed by our usual standards, but those would generally come down to a matter of research. Stack Overflow is not in a position to be a peer-to-peer research platform. There will also be questions in which a consensus for their topicality will not be reached, for the reason once said in this question about pure machine learning questions:

There is rarely a broadly applicable rule of thumb for these types of things. We'll only end up writing some rule that gets religiously misinterpreted and innocent by-stander questions get hurt.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .