tl;dr: While there is no clear-cut rule of thumb to this, we might be able to, in most cases, educate our users on whether it belongs here or somewhere else, and ultimately what they should bear in mind when asking. MCVE's in machine learning (and even more so in deep learning), are very hard to make.
I've observed the machine learning environment around Stack Overflow for a while, and noticed that a significant share of questions indeed do not seem to belong here. This includes deep learning questions, which is another beast in itself: due to the unique concept of neural network design, deep learning may be perceived as something distinct from traditional machine learning methods, to the point of resembling an engineering task. This answer will have a special focus on deep learning, but the advice should apply to other machine learning questions nonetheless.
In the absence of a migration, what can we say to new users in the way of guidance so that their question can be improved?
This can be a bit.. uh, misguided? ^_^ It's true that we might need to communicate our position better, but a fair share of these questions are not a proper fit in the first place, and our users will still have to be told so and why they are. The question linked above is clearly not appropriate. So do many others that show up regularly on the site (example). When thinking whether or not to close a question related to machine learning, we admit the same standard reasons for closure as for any other question. In particular, a fair bunch of questions in this field tend to come from clueless users asking for "some pointers", making them too broad for Stack Overflow. With that aside, what makes the subject more confusing than others is their general topicality, and whether we can treat them as programming questions.
So, is it programming or not?
We're living in times where machine learning is interestingly ubiquitous, so much that people may just mix all concepts in computer science together to create this amalgamate of ideas related to computer programming. Some (often sensational) magazines and blog posts have employed the term Programming 2.0 when referring to the use of artificial intelligence algorithms to solve problems with a computer (a term which I personally reject and despise). However, we can, and should define boundaries based on the fields of expertise that are involved. As much as one would like to split the waters, I believe that there is no clear consensus in the Stack Overflow community on this, but my current stance is the following:
- First, it's safe to say that we prefer not to address purely theoretical matters. For such questions that do not seem unsalvageable other than that, I usually place this custom closure message:
I'm voting to close this question as off-topic because it's purely specific to machine learning rather than programming. You may visit Cross Validated or Data Science SE instead.
It gets harder when a concrete machine learning model is presented, followed by a question on how to solve some concrete behaviour or how to improve its evaluation metrics. Questions seeking advice on "How do I improve this model?" are usually open ended, and will often attract generic recommendations that involve much more concepts than those in the scope of the site. They should be closed as Too Broad until they are severely narrowed down to a concrete concern. Providing advice here is tricky unless you are confident in proposing migration.
If the question presents a very precise problem statement and is (at least close to) a proper MCVE that reproduces the problem, then it might just seem to be worth improving. However, your mileage may vary, because such a question may still require an expert in machine learning to be well answered.
It's also worth noting that there are machine learning frameworks which do not require writing any code (example: see caffe and its questions that use the built-in neural network trainer). Models and processing pipelines may be described using other textual formats as data, which in turn are fed to a program. Is this programming? I feel that this is a bit subjective, and one can make arguments towards both ends.
What would be an MCVE here?
Let's suppose that a decision as made, and that we do want to keep it here. It's very important to understand that making an MCVE for a machine learning question is awfully hard. Minimalism is not very hard to achieve if familiar frameworks are used (such as Scikit-learn, Keras, TensorFlow, ...), but the variables which contribute to reproducibility are more than what most think of:
- Some questions only provide the model and forget about the optimization process of that model. When training the model through stochastic gradient descent, this includes the kind of optimizer, the learning rates, and related hyper-parameters.
- Even different frameworks or framework versions may yield different results, so that should be explicitly specified.
- Since the training process often has a stochastic nature, it may rely on a random number generator. Without fixing all PRNG seeds, the specific issue presented by the asker might not always occur.
- One part of an MCVE that may be hard to state here is what makes an expected outcome. Most other programming questions have a well known positive output, but the ideal solution of "making this classification 100% accurate" is unrealistic. In machine learning (or even data learning as some people call it), the quality of a model is only as good as the data we feed it. As already described above, this kind question may fall prey to the "no concrete problem statement" trap, leaving everyone disappointed. We should be able to solve that question which applied a soft-max non-linearity twice, but this is likely to be phrased in the question as an "exaggeratedly low accuracy", or "loss not decreasing", which could be anything until we look at the code.
- Most critically, unless you are using public data sets, this might not be reproducible by someone else at all. Questions with the title "CNN predicts only '1's" is not uncommon, but if they need to give us their 1 GB data set of images that they should not be disclosing anyway due to signed data end user agreements, then we're close to a brick wall. The asker can only be advised to try changing the problem to another one (e.g. try the same architecture and processing pipeline with common public data sets such as MNIST) and see whether the issue persists.
I once provided a bit of guidance on how to curate these questions. A similar process can be laid out for asking a question: If the question effectively narrows down to a programming problem, you may then put this matter aside and use the remaining criteria on whether the question is suitable for this site. If it's purely about machine learning rather than programming, please visit another site. And finally, the points above should be kept in mind when creating an MCVE. Yes, it is hard, but how can we make the question useful to future visitors without it?