44

This week, we are launching a new site for Emacs questions. It's been pointed out that there are already 11k questions tagged on Stack Overflow. Setting aside the question of whether new sites will benefit from having a more focused community of users and an expanded topic, will splitting Emacs off result in better questions and answers?

For the record, we've already spun off a handful of topics, including:

  • TeX/LaTeX
  • WordPress
  • Drupal
  • Blender
  • Joomla
  • Magento
  • Tridion
  • Mathematica
3
  • 4
    Also, SE has created Code Review and Code Golf because SO didn't want those types of questions.
    – gunr2171
    Commented Sep 23, 2014 at 16:39
  • 4
    Does it pay to spin off sites is an interesting question. It might have different answers, depending on how you look at it: As moderator, answerer, asker, lurker or google-visitor, on SO resp. the new site. Commented Sep 23, 2014 at 20:50
  • And why does Mathematica have its own site but Maple doesn't?
    – qwr
    Commented Aug 21, 2018 at 19:39

2 Answers 2

26

"Design Lessons from the Fastest Q&A Site in the West", a 2011 study of Stack Overflow, suggested several measures of Q&A success:

  • Number of active users,
  • Percent of questions answered,
  • Speed of answers, and
  • Question views.

Here is the current Stack Overflow baseline for some of these measures:

Tag/Site         questions views score closed answers accepted answer TTA
                          median   avg      %     avg      avg      %   *
-------------------  ----- ----- ----- ------ ------- -------- ------ --- 
Stack Overflow     8032148   248   1.6    4.5     1.7     57.5   88.4  24 
 

TTA stands for time to answer and represents the median time from asking to the first answer of open and non-negatively-scored questions since Jan. 1, 2014. All other statistics are for all time. Below are matched pairs of tags and sites. I extracted the numbers from public data so you can add your own site or tag to the list. Tag are listed first in [lowercase] and sites are Capitalized.

[blender]              843   238   1      4.4     1.1     45.3   77.3 601
Blender.SE            4496   139   3.3   10.7     1.3     57.3   86.7 101
[drupal]             14676   346   0.9    2.8     1.5     53     90   118
Drupal.SE            42867   175   1.1    4.2     1.3     46.6   81.7  74
[joomla]             11112   258   0.4    4       1.4     45.3   86.2 115
Joomla.SE              806    54   2      2.9     1.7     58.7   94.7 104
[latex]               3437   658   4.4    8.9     1.9     63.4   89    52
TeX/LaTeX.SE         71579   266   5.5   11.5     1.4     63.5   86.8  54
[magento]            25961   369   0.8    2.3     1.3     44     82.4 110
Magento.SE           12131    90   0.7    5.7     1.2     43     82.3  55
[tridion]             1144   222   3.5    3.9     1.9     69.5   98.4  44
Tridion.SE            2386    98   4.6    2       1.7     52.2   96    71
[wolfram-mathematica] 2993   360   4      2.2     1.9     66.7   92.5  51
Mathematica.SE       17724   186   4.8   10.5     1.6     57.7   86.1  62
[wordpress]          59188   132   0.4    3.6     1.2     42.8   80.4  33
WordPress.SE         49355   187   0.8    6       1.3     51.5   85.7  61 

Between them, these topics have more than 100k questions on Stack Overflow. It wouldn't be fair to say that the tags are abandoned; many people monitor several sites. In fact, by splitting of sites that non-programmers use, we might be increasing the overall number of contributors. That depends entirely on the new sites posing questions that would be not be answered on Stack Overflow. (Note: with the exception of , none of the tags I sampled actually have a higher close rate than average on SO.)

In general, the new sites do slightly better in terms of questions answered and speed of answers. These are encouraging signs. But answers are helping fewer people, since the questions are less visible when not hosted by Stack Overflow itself. With the exception of WordPress, the best way to get your question seen is to use an SO tag. However, the sites we spun off so far tend to address a slightly different audience than the core demographic of programmers. TeX and Mathematica are used by mathematicians, Blender by 3D animators, and the CMS sites are for authors and publishers. Therefore, these are not so much topic splits as topic expansions.

This is why I emphasized that an Emacs site needs to go beyond programming. Considering that Emacs is a programmable editor designed by and for programmers using a programable programming language, that's a tall order. But we think it's worth a shot in private beta.

For reference, here are the stats for Emacs tags and the upcoming Vi/Vim proposal:

[elisp]               2884   160   3.2    1.9     1.9     77.8   97.1  55
[emacs]              11505   227   3.6    3.3     1.9     68.9   93.5  68
[vi]                  1087   289  11.1    6.9     2.9     71.7   96.2   9
[vim]                14637   260   5.1    4.4     2.1     73.6   96.1  22

These tags are already well served on Stack Overflow, so we will be paying close attention to the sorts of questions asked during private beta. If it becomes clear that the sites merely split rather than expand the audience, we will do what's necessary to reverse the split.

13
  • 12
    But answers are helping fewer people, since the questions are less visible when not hosted by Stack Overflow itself -- This assertion seems... subjective? True, there are more people on Stack Overflow, but there are also a lower percentage of people specifically interested in the subject domain of the more specific site (by definition).
    – Robert Harvey Mod
    Commented Sep 23, 2014 at 17:08
  • 3
    I would assume that's a reference to the "median views" column, @Robert - compare the views these questions get in their SO tags to what they get on their dedicated sites.
    – Shog9
    Commented Sep 23, 2014 at 17:09
  • 5
    People view questions for all sorts of reasons unrelated to the specific subject matter. Stack Overflow gets many hits from folks (including myself) for reasons of moderation having little to do with the specific subject matter. If anything, I'd weigh more heavily those views on the specific site. Those folks are there (more) specifically for the subject matter.
    – Robert Harvey Mod
    Commented Sep 23, 2014 at 17:11
  • 1
    Sure, @Robert. But in most cases, the difference between views in the SO tag and views on the sites are not... subtle. We're a bit beyond numbers that can be explained by a few moderator views or even the views a question might get via review or the home page. Remember, the vast majority of all views come from search results - therefore, it's pretty safe to assume that folks are finding SO questions via search a lot more often than they're finding questions on individual Stack Exchange sites.
    – Shog9
    Commented Sep 23, 2014 at 17:13
  • That said, the WordPress site/tag hint that at least part of the discrepancy here might be due to individual sites hosting more long-tail questions - would be interesting to build a histogram for this.
    – Shog9
    Commented Sep 23, 2014 at 17:18
  • Hmm, this is a potentially dangerous game. When does the C++ community get their own site?
    – Cody Gray Mod
    Commented Sep 23, 2014 at 20:37
  • 1
    Shouldn't views be normalized with respect to time? Especially in the case of newer betas - take Joomla.SE as an example: the average views per day of all questions on Joomla.SE appears to be almost identical to that of [joomla] questions on SO, unless my T-SQL is funky (which it may be).
    – Air
    Commented Sep 23, 2014 at 20:43
  • It's interesting to see that nearly all listed sites close a considerably higher percentage of their questions than SO. That's probably partially responsible for their better scores on other metrics. (Another big reason, which feeds into higher percentage of closing too, is they are smaller. I guess.) Commented Sep 23, 2014 at 20:45
  • @AirThomas: That's one of the advantages of using median rather than average: less need to adjust for variables such as time. If you look at the study I linked to at the top of the answer, you'll see evidence that most questions get most of their views within a week or two of asking. So while there is a slight bias toward Stack Overflow, it probably isn't significant. You could get close to eliminating the systematic error by limiting questions on SO to the life of the corresponding site.
    – Jon Ericson Staff
    Commented Sep 23, 2014 at 21:30
  • @Cody Gray: It won't. On Area 51, I wrote: "If this had been language split (Ruby, Haskell, Lisp, etc.) we would have shut it down immediately." In addition, C++ doesn't have a leg to stand on, statistically speaking. It's the very definition of "well-served by SO". (I'm fairly down on Vi/Vim, by the way.)
    – Jon Ericson Staff
    Commented Sep 23, 2014 at 23:42
  • I hadn't read that Area 51 post, but that just complicates the issue. It looks like you were most compelled by the fact that the proposal attracted a large amount of support in a short window of time. And I'm just making this bald assertion without any real proof, but I imagine that a C++ site would gather a significant amount of support. Or if you don't like C++, what about C#? I mean, I completely agree with you about language proposals needing to be shut down as duplicates of Stack Overflow. But I don' get why they're exceptions to your general claim it pays to spin off sites.
    – Cody Gray Mod
    Commented Sep 24, 2014 at 8:53
  • 2
    @CodyGray: Because there is no chance that a C++ site could include a substantial proportion of non programming questions. And we don't know yet whether or not that's true of Emacs either. I anticipate spending a few hours between now and Monday totaling up the number of questions in private beta that could not be asked on SO, because that's what we'll evaluate to see if the site is really expanding the audience. We don't have to do that exercise with C++ or C# because it's obvious those topics would spilt SO's audience with no chance of adding to it. Emacs is an edge case.
    – Jon Ericson Staff
    Commented Sep 24, 2014 at 14:47
  • 1
    I've been playing with data explorer to try to get a more detailed picture of the relationship between views and question age, because "most views come early on" and "most views come from search engines" don't seem entirely compatible. Here's an example of my attempt at plotting average vote counts over a frequency distribution of questions by age, in case anyone cares to explore that line of thought in more detail. (The graph does get noisy as age increases and bins start to empty.)
    – Air
    Commented Sep 24, 2014 at 23:22
17

This question makes me sad. It makes as little sense as a couple whose marriage is on the rocks, but decides that sitting down like mature adults and making some changes is "too much work," so rather than go to a marriage counselor, they visit an accountant and request data to show that it would be more financially profitable if they just got divorced.

Similarly, there are three choices here. Two of them are "less work" but end unhappily and the third choice is "more work" but is the only one that has a chance of ending happily.

  1. Continue business as usual with a platform that has reached its scaling limit, perhaps causing quality to decline, and certainly having trouble attracting new users to replace the discouraged experienced users.
  2. Divorce the platform into dozens of tiny non-interacting communities, each of which will be "happy" because they are using a platform which is technically sufficient to meet the needs of their small community.
  3. Actually sit down and do the hard work of figuring out what the current scalability problems are with StackOverflow, proposing solutions and implementing them.

@JonEricson's question is equivalent to asking "show me some hard data about whether choice (1) or (2) is better," without considering choice (3).

Obviously the analogy to marriage only goes so far. StackOverflow and Careers 2.0 are a business not a marriage, and so if the long term profitability of your company are best with choice 1 or 2, then, by all means, that's what you should do. But it is difficult for those of us outside your organization to see what cost-benefit analysis you have done on option 3, and thus, externally, it appears you haven't really given option 3 sufficient consideration as being, perhaps, the most profitable direction.

The Area 51 Q&A about the Emacs proposal brought out several straight-forward platform UX bugs that might be relatively easy to fix.

  • In the question Why do we need a separate site for Emacs it was clear that one UX problem is the lack of tag and search/sort/filter/grouping capabilities that cross the existing sites. Emacs questions (like questions on many topics) are spread across SO, SU, U&L and Tex (in proportions approximately equivalent to the proportions of traffic that each of those sites receive.) There is no easy way (or at least no easy-to-find way) for Emacs experts to see all the newest or most active Emacs question across all the sites in one place without a bunch of noise from non-Emacs questions. Additionally:
    • High rep users of the emacs tag on one of those sites may (or may not) have equivalent rep on any of the other sites (thus review queue rights, let alone mod rights).
    • It is impossible to mark duplicates cross site. (Thus garbage accumulates).
    • It is non-trivial to migrate cross-site.
    • Cross posting is illegal.
    • Each site has different tag wikis, different ideas about what is on and off topic, and different pages to see who the high-rep tag users are.
    • chat rooms are associated with sites, so even if an "Emacs" chat room were started on SO, it is not clear that the users who accidentally find SU first would be able to find the SO Emacs chat room.
    • there is a site (Area 51) to propose new break-offs of SO, but no site for SE users with similar interests to find each other (an Area minus 51, as it were.)
  • In the answers to What to expect in the Emacs private beta it became apparent that there is a perception that much of the closing as "off-topic" is done in the review queues by users who do not have high-rep in the tag in question. We can all tell the difference between a question that is too-broad and one that is not, but perhaps the "off-topic" close reason should be restricted to users with sufficient tag scores for some tags?
  • It seems useful to me to step back and ask the question, "what is motivating people to propose new splinter sites, and then join and work to make those new splinters succeed?" (And sometimes they succeed magnificently.) Is it just that sometimes we need to expand the scope or audience of a tag or two? If so, isn't there some relatively easy fix for that problem other than starting an entire new site? I'd suggest that there's a stronger motivation than scope expansions, though. For better or worse human beings naturally like to form teams (or cliques) that come together and take pride in cooperatively building something together. I suspect that most of the activity on Area 51 is actually a pent-up demand for some kind of way of forming teams or other cooperative expert sub-communities on SO. There have been some proposals for adding some relatively small social networking capabilities (and also this) that might be a more effective way of addressing that demand.
4
  • 1
    +1 for analyzing the software problems that need solutions to deal with a split. We're already experiencing these with the MSE/MSO split.
    – Bergi
    Commented Sep 24, 2014 at 18:04
  • To be fair (to myself) I agree that the correct solution is to make sub-communities work on SO and other SE sites. That should have been done several years ago, in my opinion. Our initial discussion about the Emacs proposal was literally whether to tell them to come back in X months and see if the work we are doing to solve the sub-community problem has paid off. We still might do that when the Emacs private beta ends. But that's pretty rough because, as you say, there's a ton of work to be done between now and then. We don't know how long it will take, so we let the Emacs site launch now.
    – Jon Ericson Staff
    Commented Sep 24, 2014 at 18:38
  • I was with you until the last few sentences. "Listen to what your users are actually saying" implies a consensus that does not exist and/or an obligation on the part of the staff to make their site what everyone else wants it to be.
    – Air
    Commented Sep 24, 2014 at 18:55
  • @AirThomas: you are absolutely correct. I derailed my point at the end there. I've completely rewritten that last bullet and it hopefully now gets across the more constructive point I was actually trying to make. Commented Sep 24, 2014 at 21:49

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .