I wonder if this would be possible using the data that SO produces. There is a common C# question "null reference exception" (with various flavours etc). Someone created a lovely answer to cover this and nowadays the answer is almost always closed off as a duplicate manually.
When asking a question, the "Questions that may already have your answer" section is useful and people need to make more use of it. However, this clearly is ignored or disregarded with at least a decent frequency on the question I use as an example above.
My questions are:
- Using title content, question content, tagging, and the "Questions that may already have your answer" section, can we determine if a question has a high likelihood of closure because of duplication?
- If so, say a question achieves a "this is 80% likely to be a duplicate", instead of offering a list of links to questions that may contain the answer, could we push the user down a forced track of reading 1-3 questions before being allowed to post it? This information can be encoded into the question so the community doesn't just then close it off without first seeing what they went through.
The "forced questions to read" could be weighted based on the question that gets targeted as the duplicate for the relevant question, or upvotes on answers, quantity of answers / discussion etc.
A mitigating factor was the change to allow gold badge people to close off duplicates - this helps.
Curiosity causes me to ask if data mining could allow a slight advancement in the automation of duplicate detection before it falls back onto the community to do the clean-up.
Update 1:
Lattyware had a good suggestion:
If there is a really high match, we present the best match as 'Your question has already been answered, here it is!', and then give it. Below that there can be a 'This hasn't answered my question.' which removes the auto-dupe. That way it's shown as a fast track as opposed to a barrier, making it far less likely the user just skips by it.
The idea remains the same: the necessary backing metrics to be able to produce a confidence value on the question you present or the wording you use ("Your question has been answered!" vs "Your question seems very similar to this one, please take a look.").
Update 2:
l4mpi raises good points about how this aims to improve on the status quo. I think the take away from the discussion is that we don't understand the reasoning behind a user posting a duplicate even though SO already presents a comprehensive and nicely formatted list of similar questions:
- Did they see it and deliberately ignore it?
- Did they even see it? Not many standard forum sites have this feature.
- Did they see it and disregard it because they assume their question is unique in their context?
- Did they not want to leave the "Ask Question" workflow to read the questions for fear of losing their question content?
- Did they not want to spend the time reading answers that may have no relevance?
LINQ
questions suffer from this problem because there are so many variables. No 2 questions are exactly alike, but many about the same things and are probably duplicates. New users are used to forums where you are often given customized solutions and not forced to think for yourself.