Some of you may recall an experiment I ran a month or so ago, the "Answer Similar Questions" hook. It didn't work quite like we'd hoped, but the data suggest there's some promise there.
The quality of the search in the experiment was quite poor, which was noted both here on Meta and in subsequent analysis of the results. I believe before anything else is tried, we have to nail down a better search.
Rather than just make our best guess search and then subject everyone to it test it, we're going to try something we've done before and tap Meta's answering know-how to build it.
Help us build a good "Find the Next Question to Answer" search
I've thrown together a little page that lets any logged-in Stack Overflow user twist the knobs on our search service. The goal is to produce a search which, given just your recently-answered question, produces a small list of questions that you might want to answer.
The query builder interface
We're using Elasticsearch here, so those of you familiar with it will recognize a lot of these options:
- Question/Answer Link - indicates the question to use as input, and isn't strictly part of the query (it won't be saved).
- Like Query - choose the method Elastic uses to find similar posts
- More like this - corresponds to a more like this query. It is based on question titles, with various analyzers applied.
- Fuzzy like this - corresponds to a fuzzy like this query. It is also based on question titles.
- Tag Impact - how to consider tags.
- Require primary tag - all results must have the first tag on the provided question.
- Boost any shared tag - the applies a boost for each tag in common with the provided question.
- Require primary tag and boost other shared tags - a combination of the two prior. The primary tag is required, and any other tags in common with the provided question get a configurable boost.
- No older than - will filter out any questions older than the given number of hours (the rationale being that maybe old unanswered questions are unanswered for a reason).
- And finally the Additional Ranking Function - this takes an infix function (basic operators
+
,-
,*
,/
, function calls, and grouping with()
; no variables or anything fancy), which is calculated for each question, and then multiplied with the standard Elasticsearch document score. It corresponds to a function score query.- A set of parameters, using a
@
sigil, are also provided for use in the Ranking Function: those prefixed byAQ
refer to the passed/"just answered" question and are constant for a query's run, while those prefixed byUQ
refer to the under-consideration/"unanswered" question, and will vary. - There are 7 more parameters which are special cases:
@DefaultSearchScore
is the score Elasticsearch would assign to the document in the absence of the ranking function.@UserAnswerScoreInTagsOnAQ
is the sum of scores (upvotes - downvotes) the user has on other answers to questions with any of the same tags as the provided question@UserAnswerScoreInTagsOnUQ
is the same, but for the questions being searched@UserAnswerScoreInTagsCommon
is the same, but for the tags in common between the provided question and the questions being searched@UserAnswerCountInTagsOnAQ
is the count of all answers the user has on other questions with any of same tags as the provided question@UserAnswerCountInTagsOnUQ
is the same, but for the questions being searched@UserAnswerCountInTagsCommon
is the same, but for the the tags in common between the provided question and the questions being searched
- A set of parameters, using a
These are the knobs we think matter most, but if you can make a case for another parameter or option, please do so as a comment to this question! Note that some things won't be possible due to how we index questions.
The following constraints that are applied to all queries:
- Because we're only trying to surface a few questions as options, this page only returns the top 5 results of the search
- Questions must be in a "normal"-state, not locked, closed, deleted, etc.
- Questions must have a score >= 0
- Questions must be unanswered (have 0 undeleted answers)
- For safety reasons, there's a throttle (1 query every 30 seconds), but since I've been evaluating each question for its "answer-ability" I haven't found that too onerous.
Please save any promising queries, and post links and details as answers to this question. A link to a query, saved from the example show above.
Update:
this interface has been disabled, the winning(-est) query was proposed by Travis J in this answer. Analysis showed an increase in second answer rate, and that the answers so posted were typically better (as measured by reception w.r.t. voting).
The actual "next answer suggestion"-feature remains disabled while final adjustments are made (exposing the dismissal setting, tweaks around answer flow, changes to prompting logic, etc.). I'll probably be posting again in the coming weeks with the final-ish design.
serialization
for example, and resulting on unwanted questions like apython
one (since my main tag would actually bejava
). It could be workarounded with require main tag, but I think that boosting shared tags option is better, besides that side effects.