6

When a post is flagged as a duplicate, is that (manually created) association factored into the suggestions pane when crafting a question? If not, can that be added? For example:

User A asks question A. Question A is a semantic duplicate of question B.

While these two posts are now intertwined, I propose that syntax and semantics of the questions should be compared and studied, and used to assist in the automatic suggestion of questions. The idea is, in order to train the suggestion software, you need to provide golden path or "correct" information. Since the posts that are duplicates actually are duplicates, they can serve as a base case for guessing if a new post is one.

Since it was asked, here's a VERY primitive example I whipped up in a few minutes. First, some background. Computers, like us, can determine with a degree of certainty the part of speech being used in a given sentence. As a primitive example of what I mean, here are two question titles that are duplicates, but are structured differently.

I use the Penn POS set which you can find here.

[('How', 'WRB'), ('do', 'VBP'), ('I', 'PRP'), ('commit', 'VB'), ('all', 'DT'),
 ('deleted', 'JJ'), ('files', 'NNS'), ('in', 'IN'), ('Git?', 'NNP')]

And now for the duplicate:

[('Removing', 'VBG'), ('multiple', 'JJ'), ('files', 'NNS'), ('from', 'IN'),

('a', 'DT'), ('Git', 'NNP'), 
('repo', 'NN'), ('that', 'WDT'), ('have', 'VBP'), ('already', 'RB'), 

('been', 'VBN'), ('deleted', 'VBN'), ('from', 'IN'), ('disk', 'NN')]

This is the most entry level analysis, so there's not a ton to extract here.

Next time you see a question being inputted with either sentence structure, look for questions that have similar keywords ('delete/d', 'Git') - be very relaxed on the search. Obviously, if you bring up questions just about Git, you'll get a lot of irrelevant results.

Possibly, however, if you look up questions that are about Git with that sentence structure you may be able to find questions similar in semantics.

Again, I must stress that this is an incredibly basic analysis. It just serves to explain what I'm driving at.

Here's what I'm referring to.

9
  • "Is there potentially meaningful information to mine in providing the manually verified data?" Likely. Commented Jun 30, 2016 at 20:21
  • 6
    What is the feature request? I find your question a bit hard to follow
    – rene
    Commented Jun 30, 2016 at 20:22
  • @rene I have clarified what I was asking.
    – Athena
    Commented Jun 30, 2016 at 20:56
  • @gevorg I edited it to format it as a feature request.
    – Athena
    Commented Jun 30, 2016 at 20:57
  • 1
    Why would SO spend time and money on this when they have actual humans (who are still better at parsing language than computers) doing this already? And they don't even have to pay the humans! Commented Jun 30, 2016 at 21:09
  • 2
    Why would SO spend time and money on any features like these? Because they improve the overall UX for everyone involved. Dealing with the vast ocean of duplicate questions being asked (heck, this might even be a duplicate) is a useful investment.
    – Athena
    Commented Jun 30, 2016 at 21:12
  • 2
    You may be interested in the SO Python chat room's Project Nidaba; you should certainly drop by the room & chat with the Nidaba contributors.
    – PM 2Ring
    Commented Jul 1, 2016 at 16:36
  • 2
    Dupe matching is not easy, even for humans. FWIW, when I'm searching for good dupe targets I try to find a decent match between the new question and the old one, but IMHO it's more important that one or more of the old answers are applicable to the new question, even if the questions themselves may not appear on the surface to be a close match.
    – PM 2Ring
    Commented Jul 1, 2016 at 16:42
  • People frequently mark questions as duplicates when they're really not, so then there are invalid links between questions that really are only superficially similar.
    – nnnnnn
    Commented Jul 3, 2016 at 1:22

1 Answer 1

3

I think it is a good idea to have automatic syntax and semantics studying and doing suggestions based on results.

It looks like only the title is being used now.

Look what I got, when I tried to ask question with title similar to one of the duplicate questions.

my screen

Duplicate questions are using the special tag there, so that is how the votes are being used currently.

8
  • Ah, but this is comparing user input provided against the input in existing questions. There is no semantic link there - the question has already been asked. But suppose I ask a question that has a pattern similar to a question that was asked and closed. Even though that exact question has not been asked before, a question that followed that pattern was asked, and ultimately flagged as a duplicate. In other words Can we use the community's input on what's a duplicate as a tool for automatically detecting future ones?
    – Athena
    Commented Jun 30, 2016 at 21:00
  • I think it might be possible, however it looks to me complicated as duplicate flag is being approved by multiple users, so you need a way to combine/merge that input.
    – gevorg
    Commented Jun 30, 2016 at 21:05
  • 1
    Well, a question has either been successfully closevoted as a duplicate or it hasn't. The real trouble would be in the analysis of the questions, I would think. It's not impossible. I can say that because on my other monitor is an NLP suite I wrote...
    – Athena
    Commented Jun 30, 2016 at 21:07
  • I guess makes sense to share some sample in that case, because it already looks interesting :)
    – gevorg
    Commented Jun 30, 2016 at 21:12
  • I have included a primitive example using off the shelf tools. Hopefully that helps clarify what I mean.
    – Athena
    Commented Jun 30, 2016 at 21:28
  • @Ares I got your point and like it, look updated answer, seems now suggestions use only title.
    – gevorg
    Commented Jun 30, 2016 at 21:30
  • @gevorg - Multiple users aren't needed to close as duplicate: users who've earned a gold badge for a particular tag can single-handedly close questions with that tag as duplicates.
    – nnnnnn
    Commented Jul 3, 2016 at 1:28
  • @nnnnnn thanks, I was not aware about that, usually I see multiple names under the close message.
    – gevorg
    Commented Jul 3, 2016 at 7:08

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .