14

Do you think a mechanism, maybe AI running in background, could help moderators to identify duplicated and/or related questions? Does it already exist in SO?

9
  • 6
    There's already a relatively decent algorithm that finds possible duplicates when the question gets asked, based on tags, keywords, and votes, I think. The "related" column to the right is also a good place to look. Other than that, yes, some form of advanced automated system could probably help, but without a concrete and specific implementation (or implementation suggestion) I don't quite see the point of this question? If you have a suggestion to make, I suggest you expand your question.
    – Pekka
    Feb 9, 2017 at 11:58
  • 3
    @Pekka웃 I've being studying this subject and I'm noticing that even with all the good mechanisms to prevent the user to ask, many of them just ignore these mechanisms. I'm thinking of working in an implementation of it and make it available for the community. The intent of this question is just for assessing the feasibility.
    – Digao
    Feb 9, 2017 at 12:03
  • 2
    You mean whether it's technically doable, or whether it would make sense to do? As to the latter - yes, I'd say that could be a very cool thing to have as a browser extension, for example.
    – Pekka
    Feb 9, 2017 at 12:04
  • 2
    Related meta.stackexchange.com/q/286329/213575
    – Braiam
    Feb 9, 2017 at 12:05
  • 5
    There are some community projects underway. I made some of those involved aware of this question.
    – rene
    Feb 9, 2017 at 12:06
  • 1
    Like @rene mentioned, there are atleast 2 that I can recall, 1. github.com/ArcticEcho/UniStack 2. github.com/yvettec/Dupes ... Feb 9, 2017 at 12:09
  • @Pekka웃 I mean if it makes sense, but looks like this endeavour is already underway...
    – Digao
    Feb 9, 2017 at 12:16
  • 2
    Perhaps this should be tagged as discussion instead of feature-request
    – user247702
    Feb 9, 2017 at 14:09
  • 3
    @Stijn done as I think that is the correct thing to do ...
    – rene
    Feb 9, 2017 at 14:21

2 Answers 2

39

AFAIK, this is what SO users (not SE) are doing.

Current situation

Ada

This project was started by Yvette using the proof of concept algorithm. The bot finds by itself the duplicates comparing to other posts (es. nullpointer exception) and reports in chat., Yvette got busy, and the project got stalled.

UniStack

This project was started by Sam, using various NLP techniques. Sam pushed on and made some progress, but the algorithm remains unfinished as he's been busy with other stuff. We have done some initial testing, but no conclusion has been reached yet.

SOCVFinder

This bot is in production, it reads all comments posted on SO and this way are able to notify users (normally gold badge holders) when another user have suggested possibile duplicate. The bot runs in multiple rooms and can either just output chat message when "possibile duplicate" is found or users can opt-in to get pinged. Furthermore the bot can be used to search on old duplicates, hence create a batch with previous duplicates in desired tag. Currently this bot has around 2000 confirmed duplicates closed.

FireAlarm-Swift

This bot is also in production but it is not targeting directly duplicated post, just low-quality question using machine learning.

Would SO community benefit

If a question is a duplicate I think the whole community agrees that it should be closed as such. Furthermore informing dupehammer users (gold badge holder for that tag), reduces the reviews needed to close a question as a duplicate.

However, SO's close vote review queue is not working (too many posts to review and too few people reviewing). Currently a lot of close votes just age away. Furthermore, the dupehammer users are not encouraged them to use their single-duplicate-vote close powers. Often they only receive complaints and revenge downvotes from the people who disagree with closure.

Considering this, it is difficult to find people that are willing to review tons of stuff. It can even be counter-productive to push even more questions in to the review queue because it spreads the few existing reviewers' votes thin across more posts, which can lead to even more close votes being wasted and fewer posts actually accumulating the 5 votes needed to be closed.

Conclusion

If you have fun developing and testing bots related to moderation on SO, your are welcome to the SOBotics room, we are happy to share our code (it's all open source) and host you in room if you like to test.

For more information about the room and all the other moderation bots running in it see sobotics.github.io

6

I haven't done an extensive search to see whether anything comparable to this has been suggested before, but if there isn't automation available for handling duplicates, what about letting users add questions to their own personal list of 'canonical questions' — rather than making them rely on bookmarks?

For example, I have a list of maybe 50 bookmarks for 'canonical questions', but it is something of a nuisance because I keep my bookmark list in a fairly narrow column, so it is hard to make the titles snappy and mnemonic. If I could instead jump to a personal list of 'canonical questions' grouped by tag, and with the ability to annotate what the question is about, it might make my life simpler. Ideally, there'd be some sort of expandable or collapsible display, and a question might be canonical in multiple tags. People would be able to remove questions from their list too. I'm neutral on whether the questions must be listed with their current title or whether the user should be able to specify a 'title'.

For sub-10K users, deleted questions would effectively be removed from their canonical list because they can't see deleted questions. For 10K+ users, deleted questions may once have been appropriate; they should be clearly displayed as deleted, but the information should not simply be thrown away. Going to the deleted question may suggest an alternative canonical question.

Of course, there'd be an easy way to get the link to the question so that designating a duplicate would be as simple as possible — like the 'share' links, for example.

I think I'd be happiest myself if the 'select duplicate' option popped open a new tab in the browser for me to work with, rather than a popup within the current tab (showing the question to marked as a duplicate). I'm sure other people with more UX knowledge will have alternative ideas.

This would allow each person to have a list of canonical questions, rather than requiring the system to identify them automatically. I would like to see some automatic duplicate support — more than just the related questions list. But something along these lines might help people manage duplicates until the all-singing, all-dancing automatic system is made available. It probably is less difficult to implement, too.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .