8

Sometimes, organizations don't communicate very well internally. Over the years, we've found this to be particularly true when it comes to large software companies working on the same project, or university students working on (at least) similar long-term assignments.

What happens is we end up with quite a few questions that should generally be closed as duplicates, but volume is often a big problem with the efficacy of any system. To start chipping away at this problem, we're going to be testing a new system over the next few weeks.

What does this system do?

It keeps track of what happens to questions that are asked from what appear to be people strongly related to one another in endeavor. Network 'neighborhoodness' is a big signal here, but there are other signals that we might be able to eventually put to some use.

When we detect that folks might be throwing themselves at a wall organizationally, we have the option of:

  • Being more targeted about what goes right past triage and straight into another queue
  • What probably isn't a good candidate question to show someone that's poking through a tag feed looking for things to answer
  • Sending a "We're pretty sure people working with you just asked this whole stack of relevant questions, go look?" message to users we detect repeating questions (possibly)
  • We have to see.

What does testing entail?

We go off of signals like posts being closed for various reasons. Similar to our anti-abuse system, certain events carry more certainty, some less, some won't really show how useful they are until we look at aggregate data.

We're going to be messing with lots of knobs while we bring the system up beyond the passive looking that it has been doing.

What could go wrong?

Well, if there's a duck, plenty. But we're pretty sure the worst case possibility is you occasionally see something in review that obviously doesn't belong there. We won't be using the data this system collects to alter how questions are shown on the site in the near or immediate future as far as we know.

I have a question or saw something funky happen

Leave an answer, and we'll get right on it.

When does this start?

Monday, June 26. Ending date to be determined for now. Most people should not notice anything different while we take some time to make sure everything works. Once that happens, we'll share what we found, and our plans on going forward.

Planned for the network, or just Stack Overflow?

If it tests and works well and there's merit, we're not opposed to giving it to sites where it might help. However, this is a question we're probably not ready to answer for a while as there's still a bunch of work to be done.

17
  • 19
    This sounds interesting, but there isn't a lot of detail here. Can you explain a bit more? Is this thing going to work based on perceived user location? Content of posts? Votes provided by other users? A combination of all three (or more)? Is it only looking for duplicates (and is it using the results from work the community helped with last year)?
    – Andy Mod
    Commented Jun 23, 2017 at 13:33
  • 60
    I don't understand.
    – Suragch
    Commented Jun 23, 2017 at 13:36
  • @Andy It's going to use multiple things (also looking at what gets closed and why), and I hated to be a little vague, but part of the testing I'm doing is looking at how much of a factor each small thing is. Why was a post closed? What did voting look like? Was it from a nearby network as another post? And then again I don't want to get too specific and end up creating instructions on how to get around it, so ... I'll update my post in a little while and try to be a tiny bit more specific.
    – user50049
    Commented Jun 23, 2017 at 13:37
  • 38
    -1; I wouldn't be comfortable with Stack Overflow explicitly trying to direct my colleagues towards my posts, especially if I were in a more corporate, less friendly environment than the ones I'm used to. I've once had to carefully deal with a coworker making an edit to my post that I didn't agree with that (in my view) basically hijacked my post to bolt on an extra vaguely-related problem we were having with the tool I was asking about; he was cool about it, but in a more politicised work environment, reverting something like that could lead to ugly interpersonal conflict very quickly.
    – Mark Amery
    Commented Jun 23, 2017 at 13:42
  • 2
    So if some other Stack Overflow employee lost his keys, the system will bring him in contact with you so that you can tell him/her how you found them back?
    – Glorfindel
    Commented Jun 23, 2017 at 13:46
  • 58
    Basically, I don't think you've properly considered the "What could go wrong?" here. The potential problems aren't just for reviewers; they're also for askers. Telling people "hey, this guy (possibly with a pseudonymous account) looks like a colleague of yours and is asking the same question!" is potentially a serious privacy violation, and it risks causing askers to be pressured by colleagues or managers into doing what those people think is best for the company rather than trying to create good content and obey Stack Overflow's rules. Seriously, whatever you do, don't do that.
    – Mark Amery
    Commented Jun 23, 2017 at 13:47
  • 3
    Related to what @MarkAmery said, a user recently brought up a related concern about being able to identify a coworker over on Bitcoin.SE: bitcoin.meta.stackexchange.com/questions/824/…
    – Andy Mod
    Commented Jun 23, 2017 at 13:50
  • 3
    How would you handle university lodges, where multiple students share the same connection and might request the same questions based on their studies. What happens when askers are onsite and both he and a customer try looking for the same question? Could be a bit painful for the asker towards the customer
    – Icepickle
    Commented Jun 23, 2017 at 13:51
  • 18
    I'm very surprised this phenomenon would be a significant factor in the gigantic pile of duplicates that is stacking up... but then you have the insight, and I assume you're been running the numbers
    – Pekka
    Commented Jun 23, 2017 at 14:33
  • 1
    So, like, what is the significance of the proximity/organization/whatever? If you just consider two or more people anywhere asking similar questions at similar times, why not do the thing that's looking to be done? Seems all the benefits still apply. Is there some potential endgame you're just not ready to talk about yet?
    – aw04
    Commented Jun 23, 2017 at 21:13
  • 8
    I'm stuck as to voting up or down. This, obviously, has taken significant developer time. From what's described, it might provide some marginal improvement to SO (vote-up) (ignoring the potentially significant pitfalls). My problem is: the benefit/cost ratio seams much lower than other things that could be done as improvements. Various existing RFEs would take far less developer time, but would have the potential to provide more improvement. Thus, I feel this was a poor resource allocation choice (vote-down). Please, take care of the low-hanging fruit first, or at least simultaneously.
    – Makyen Mod
    Commented Jun 25, 2017 at 4:01
  • 1
    Am I correct in thinking there's basically two aspects to this: (1) providing another signal (or signals) for the duplicate detection system already in place and (2) changing what's actually done with questions based on their likelihood to be duplicates? Or is this an entirely separate system? If yes, why? Commented Jun 25, 2017 at 16:32
  • 3
    Perhaps this is seen most frequently in the [new-speak] tag? This smacks of serious Big-Brother-ism. Introducing "features" such as this could tend to put people off the site if they find that they're being correlated with their fellow students/co-workers/fellow-rebels. DANGER! DO NOT GO HERE!!! Commented Jun 25, 2017 at 18:15
  • 14
    Why does this post seem so cryptic?
    – dthree
    Commented Jun 25, 2017 at 22:04
  • 3
    @dhtree Are you saying there's something unclear about the phrase "When we detect that folks might be throwing themselves at a wall organizationally, we have the option of... What probably isn't a good candidate question to show someone that's poking through a tag feed looking for things to answer"? Commented Jun 26, 2017 at 12:10

2 Answers 2

48

I think this feature is wrong-headed. I'm sure it could improve things on the site. But the number of duplicates due to being from the same organization/classroom is pretty small, relative to the number of duplicates on the site as a whole.

A comprehensive solution to the duplication problem, such as the one suggested here, would handle organizational duplicates as well as non-organizational duplicates. So the time spent on this project could better be spent elsewhere.

4
  • 1
    Maybe we can even get Watson to help out as well :) Commented Jun 23, 2017 at 14:20
  • 1
    Why not start small though? It sounds like the other 'deduplication' project you link to is not dead in the water, just not ready to be scaled to SO sizes yet. This project sounds like it will be a way "To start chipping away at this problem..."
    – Ben.12
    Commented Jun 23, 2017 at 17:40
  • 8
    @Ben.12 make sure to read the concerns raised in comments on the question. If there's no harm, sure, starting small is fine. If the small band-aid on a huge problem also incurs risk of privacy violations or other kinds of backlash to users, that's a different matter. Commented Jun 23, 2017 at 20:33
  • 1
    @AndrasDeak IMO those privacy concerns will be there with any duplication identification. If people from an organization/university are asking the same questions, a good dupe identifier will point that out. I would agree that "Sending a 'We're pretty sure people working with you just asked this whole stack of relevant questions, go look?'" is probably leaking too much information. But if users from the same org are asking the same questions, it's likely that they'd be able to deduce who their colleagues are anyway.
    – Ben.12
    Commented Jun 26, 2017 at 14:36
5

I see this as having applications which aren't just limited in scope to organizations or school institutions; it's very much the case that two people in opposite corners of the world can be working on a similar problem.

If this is data-gathering only, then that's fine, but I'd also like to know if there are going to be any community-oriented action items available. That is to say, if you do find anything out about this, should we start closing the questions you find as a dupe?

(As a side thought, if you did find similarities, what would be the criteria you'd use to define a "canonical" dupe?)

You must log in to answer this question.