239

As part of the Documentation Beta, we’re going to be doing a series of meta posts (and blog posts in all likelihood) with what we’ve learned, and what we’re planning to change accordingly. See the next post in the series.

These posts will normally be about once a week, but since we just launched last Thursday we’ve already got lots to share.

What we’ve learned

  1. Ho-boy, lots of contributions. We got more activity in 1 day on Stack Overflow than we did in the entirety of the private beta. Lots and lots of bugs that didn’t come up in our (months of) private testing. Accordingly, since Thursday we’ve been focused on bugs.

  2. Reputation needs work. Adam already posted some tweaks, and we’ll have more in the future. Again we tested in the private beta, and did some additional offline modeling. The basic model (rep for example upvotes, approved changes, and upvoting answers citing docs) doesn’t seem terribly broken - but the limits (like who gets rep for examples, and how much) definitely need work. Expect more reputation tweaks in the coming weeks.

  3. There’s a backlog of about 2,000 proposed changes (vs. something like 22,000 total changes reviewed). Some of this is too much content from over-enthusiastic users, some of it is presentation in the UX having too much friction, and some of it is the system limiting trusted users more than it should. There are planned changes below to address this.

  4. Some topics have way too many examples. Part of this is because many folks are just trying it out, and adding examples is easy. Part of it is that the “expected number” of examples isn’t well indicated. There’s a planned change below to address this. We will also consider capping examples per topic if this continues to be an issue.

  5. We haven’t quite figured out how to allocate numbers of outstanding changes a user should be allowed to have, and number of reviews a user should be allowed to do. For now we have simple daily caps. We’ll keep an eye on rejection rates, and what gets approved, as the “launch bounce” dies down and tweak accordingly.

Planned changes

  1. A review queue for Proposed Changes. We were hesitant to do this while Documentation hadn’t been merged into Stack Overflow, and uncertain that such a queue would be necessary. A few days of public beta have shown us otherwise, so this is now on the roadmap.

Here’s a (very early, very rough) mockup of the new queue.

Review Queue

  1. Tag merges and aliases for Documentation. We will be adding a mapping from related tags to “master” tags (like python-2.7 and python-3.x are related to the “master” python). Such a mapping will migrate existing topics to the master tag and prevent re-opening the aliased tag for Documentation. Initially we (Stack Overflow employees) will be doing these by hand; it may be opened to moderators and high rep users up in the future.

  2. A short, optional, “Focus” section at the top of each topic - sort of like what was suggested in this question. This section will be used to define the scope of a Topic and should guide contributors with respect to what sorts of examples belong on a topic. It is explicitly not a summary of the topic, but it will help readers get the gist of what they’ll find.

46
  • 61
    What about minimum tag score for approving proposed changes? Is that going to be integrated in the review queue in some way? Right now, a lot of low tag score users are getting their incorrect edits approved, as it only takes 1 approval to make it live.
    – tktsubota
    Commented Jul 26, 2016 at 2:13
  • 9
    @tktsubota minimum tag score is problematic in lower-frequency tags especially, where there are so much fewer people with that tag score.
    – bwoebi
    Commented Jul 26, 2016 at 2:15
  • 118
    @Hack-R I have 15k rep and I do not think I should be allowed to post without approval. Not because I'm not knowledgeable, but everything I add to the docs should be checked. I sometimes word things badly, break the format inadvertently or even may introduce a blatant mistake. I maybe am a core contributor to php-src and to the official manual, but this does not mean I never post mistakes. Documentation has a much higher bar of correctness than normal SO posts, IMO. If it's just simple typos, they're also quickly reviewed by anyone else, it takes barely 10 seconds to review such a proposal…
    – bwoebi
    Commented Jul 26, 2016 at 2:48
  • 63
    @Hack-R Documentation has a much higher bar of correctness than normal SO posts, IMO. Backlog buildup is in my experience not a problem in well-frequented tags. The problem are mostly these statements like "fetch() in a loop is faster than foreach over fetchAll()". Most people read that and think TIL! … very few people actually have the knowledge to state it's wrong (and even fever check it). Documentation is supposed to be far more authoritative than normal q&a is. It's maybe only a small difference in quality, but an important one. Just like 99.4% and 99.9% are only 0.5% away...
    – bwoebi
    Commented Jul 26, 2016 at 2:53
  • 8
    No mention of a better education process for all to streamline understanding of what should and more importantly shouldn't be documented. The tour and help center are thin. A decent slide presentation or video would help quash useless proposal requests, topics etc and be a point of reference for rejecting garbage
    – charlietfl
    Commented Jul 26, 2016 at 3:17
  • 14
    Love the "Edit & Approve" option. That was desperately needed. I'm assuming the idea is to only apply the additional edit if the parent edit is applied? Would that need reviewed also? Commented Jul 26, 2016 at 4:31
  • 8
    So y'all are not convinced that we need some sort of commenting or discussion system, then (as suggested meta.stackoverflow.com/q/328895 and meta.stackoverflow.com/q/329173)?
    – Frank
    Commented Jul 26, 2016 at 4:35
  • 3
    One note, just because something isn't listed above doesn't mean it won't be considered in the near future or isn't being considered now. It's just that the team has to start with a manageable set of improvements that can be implemented in a short amount of time before moving on to the next round of updates.
    – Kurtis Beavers Staff
    Commented Jul 26, 2016 at 5:22
  • 29
    I don't understand why you think a review queue is necessary. The biggest problem I've seen is not a backlog of suggestions, but rather incorrect suggestions getting approved. A better way to deal with the backlog is what you alluded to in the question: trusting trusted users more. We let 20k users edit tag wikis unimpeded; why not do the same for Documentation? Similarly, a single vote from a trusted user should always approve or reject a suggestion, clearing the queue much more quickly. I really dislike a review queue because tag experts should be doing most of the reviewing.
    – Cody Gray Mod
    Commented Jul 26, 2016 at 5:40
  • 5
    Will there be a review queue per tag? Because there is no way I am qualified to review most of the tags.
    – Galik
    Commented Jul 26, 2016 at 9:19
  • 9
    "Tag merges and aliases for Documentation." Wait a second: You declined my request for exactly this, then turn around and do it? Commented Jul 26, 2016 at 14:10
  • 23
    I just browsed a bit through Documentation and I was negatively impressed by the quality of the content. It seems that many contributors have never heard of code comments but at the same time like to copy verbatim from Wikipedia. If it stays like this, Documentation won't be useful. Commented Jul 26, 2016 at 14:30
  • 6
    @Trilarion: "But code usually contains comments and those I am missing." Coding style is not universal. There are plenty of people who rarely use comments because they try to write code that's easy to understand and therefore doesn't need them. Commented Jul 26, 2016 at 14:56
  • 2
    Trying not to kick in a possibly already open door: the only real hangup right now appears to be the overwhelming enthusiasm for the feature. You have enough, removing the banner from the front page could be sensible. You can always put it back later. Commented Jul 26, 2016 at 22:53
  • 3
    Where did all the bug/feature request posts go from the private beta? I had at least 10 that were open.
    – J Atkin
    Commented Jul 30, 2016 at 14:04

13 Answers 13

205

Note: This is now a feature request. If you agree, please upvote!

The Problem

There is an issue right now with incorrect information being added to documentation.

Why?

  • 1 rep users with 0 tag score can propose an incorrect change
  • 100 rep users with 0 tag score can accept an incorrect change
  • Only 1 user needs to accept the change before it goes live
  • It takes 2 users to determine a change is incorrect and reject it

Current Proposed Solutions

Minimum tag score or tag badge required for contributing

  • Pros: Requires some knowledge of the tag being contributed on, not filling the review queue with incorrect edits
  • Cons: Some users are left out, even if they are just starting on SO and want to contribute.

Minimum tag score or tag badge required for approving changes

  • Pros: Any user can propose a change, and users above the minimum tag score can approve it. This solves the con from above.
  • Cons: Tags with not so many users will have proposed changes sitting in the queue for days, waiting for the few people who can to approve the changes.

Minimum reputation required for approving changes

  • Pros: Tags with not so many users can get still get the changes approved. This solves the con from above.
  • Cons: Those with high reputation but unfamiliar with the topic can still approve changes.

My Proposal

Taking from "Minimum tag score or tag badge required for approving changes" above:

  • Anyone can propose an edit. This allows users to contribute who may be experienced in the topic but haven't contributed on SO proper.
  • At least one person with at least the bronze tag badge has to approve/reject the proposed change before it becomes final. This is to provide accuracy to the docs (slightly changed thanks to @4castle in the comments—see edit history for the previous revision).

As for preventing proposed changes from sitting in the review queue...

For New Proposals

  • Change the criteria for a tag to be eligible on documentation from 500 questions to 25 people with at least the bronze tag badge. This prevents dead tags like this, this, or this. See here for the number of bronze tag badges that have been awarded for each tag.
  • Out of the 5 people who commit to the proposal, one must have at least the bronze tag badge.

For Existing Tags

  • For those that meet the criteria above, do nothing!
  • For those that do not meet the criteria above:
    • If it is dead (no one is contributing or 1 person has a bunch of proposed edits that will never be approved), refer here.
    • Otherwise, I currently do not know of a solution (manually look to see if one or two of the reviewers have the bronze tag badge?).

Note: Anywhere I said "bronze tag badge" could be replaced with "25 tag score" or "50 tag score". The bronze tag badge is just a reasonable example that SO already implements and records users who have it.

32
  • 7
    Or another option, require that at least one of the people approving have a bronze tag. I also feel that the bronze tag requirement could be reduced. Commented Jul 26, 2016 at 4:13
  • @4castle "one of the people approving have a bronze tag" I think that's what I said, let me know if I could make that clearer. As for the bronze tag requirement, that's just implemented on SO already, but I guess it could be changed to 25 or 50 tag score (instead of 100). SO already tracks awarded bronze tag badges also, so it would be easy to determine docs eligibility using bronze tag badges.
    – tktsubota
    Commented Jul 26, 2016 at 4:18
  • 3
    What I meant was, say it takes 3 people to approve an edit. Only 1 of those people need to be experienced in the tag (for verifying accuracy), the other 2 people would be for making sure stuff is in the right format. It seems most edits which are rejected don't even require knowledge of the topic in order to spot. Commented Jul 26, 2016 at 4:22
  • @4castle I see what you mean, that's actually a good idea. I'll add it to my answer.
    – tktsubota
    Commented Jul 26, 2016 at 4:23
  • Technically not bad. But there though are a bunch of tags which are important should be documented, but still have not a lot of bronze-badge holders. Take for example network-programming. A very important topic. But it doesn't have 25 bronze badge holders. [not by a large margin, but still]. You are essentially proposing to exclude important tags because there are not very much questions about it. The more specific a domain is, the less questions there are. But this doesn't make it any less important.
    – bwoebi
    Commented Jul 26, 2016 at 4:29
  • @bwoebi True, we could just remove the 25 bronze badge criteria (maybe make it 5 just for dead tag safety?) but keep the requirement of at least one person with at least a bronze badge to commit.
    – tktsubota
    Commented Jul 26, 2016 at 4:32
  • @bwoebi But then again, the bronze badge is just an already-implemented SO feature. It could be reduced to 25 or 50 tag score if needed.
    – tktsubota
    Commented Jul 26, 2016 at 4:32
  • 2
    I just want to say, it has a strong bias towards large topics. The larger a topic is, the more questions there are, the faster you get to a bronze badge with little knowledge. It is quite not universally applicable. It either allows too much reviewers or not enough. 25 bronze badge holders of which 1 is active/interested in documentation is not enough. And 3.2k possible reviewers are far too much, then we could just as well allow anyone with 2k+ rep.
    – bwoebi
    Commented Jul 26, 2016 at 4:37
  • 1
    @bwoebi There could even be some implementation to check the number of bronze badge holders, and if there is too many, use the silver badge holders instead. But something needs to be tried out first before more small refinements take place.
    – tktsubota
    Commented Jul 26, 2016 at 4:41
  • 5
    @tktsubota Thinking about it, one could perhaps allow e.g. the top 1000 people with at least a 20 score in a tag or having 400+ score in a tag to be able to always review. Everyone else can also review, but their votes are non-binding. Then, when they have reviewed a consecutive 30 times with a high success rate (say 28 times out of 30 they were right), their votes become binding. I just strongly dislike the idea of flat out disallowing people to review because they're not that long on the site or similar.
    – bwoebi
    Commented Jul 26, 2016 at 4:51
  • 1
    @bwoebi Sure, I'll get started on that.
    – tktsubota
    Commented Jul 26, 2016 at 5:01
  • 1
    I like the idea of greater reviewing power with tag score, but I think it shouldn't be a hard-and-fast barrier, since that wouldn't apply well over all tags and would be rejected by the SO staff on principle (as shown on the feature request: meta.stackoverflow.com/a/329551). We could require five votes to approve, but give bronze badgers a tag weight of three; silver four; and gold five. Then have a proposal fail when it reaches a net score of -3 or worse.
    – Frank
    Commented Jul 26, 2016 at 12:12
  • 5
    We want to be thoughtful before jumping to changes too quickly after seeing how things go, but FWIW I like the thinking here. We're extremely reluctant to establish any restrictions that prevent (or significantly restrict) new users from contributing and getting involved, but SOME kind of validation on the approval side seems like a logical approach to look at.
    – Jaydles Staff
    Commented Jul 26, 2016 at 12:32
  • 4
    @bwoebi The feature request is up.
    – tktsubota
    Commented Jul 26, 2016 at 18:03
  • 1
    I am unsure on the bronze / silver / gold badge requirement idea, because this may significantly limit the number of people that would be able to contribute. Similar to how many of the people that "moderate" the site (ie. review queues, flag / editing bad posts, etc.) do not necessarily participate in Q&A as much as other people, we may see a similar thing with Documentation. The people with big badges may not be interested in contributing to the docs. I have an idea, why don't we just limit the daily rep cap to 150(ish) for documentation, instead of 200? problem solved.
    – John Smith
    Commented Aug 3, 2016 at 16:47
138

I should preface this (somewhat negative, I'm afraid) post with a comment that I've felt uneasy with the Documentation proposal from the outset—though I've hitherto reserved judgment, hoping that my pessimism would ultimately prove unfounded and it would become the "Wikipedia" of technical knowledge.

Now seeing it in action, however, I am really very uncomfortable with it indeed.

My Q&A contributions to date have primarily been in , whose official reference manual I have always found to be very thorough whilst easy to navigate and comprehend. I therefore approached StackOverflow's new MySQL Documentation with some trepidation, fearing it would at best duplicate the official resource or at worst become a disorganised mess of poorly written tutorials that fail to address edge cases.

So far, what I see is closer to the latter than the former. Granted, over time the quality of specific topics/examples should improve: but there's so much work to do, and relatively few contributors who write to a satisfactory level of quality, that I fear low quality contributions will (if not have already) become quite overwhelming—much as is already the case with Q&A. Even with sufficient high quality contributors, each tag's documentation would require dedicated curators to maintain consistency and organisation.

Indeed, we appear simply to have taken all of the issues that have been raised over quality on StackOverflow and embodied them in something that purports (or at least, strives) to be even more canonical. This is disastrous.

The only way that I can see this being (potentially) salvageable is to hugely restrict those who may contribute to Documentation. Unlike Wikipedia (which is generally only edited by people who have "reasonable" subject knowledge) I fear that many StackOverflow users—perhaps programmers in general—either think they understand their subject a lot better than they actually do, or often just post anyway about things they know they don't fully understand (perhaps in some attempt to win more magical unicorn points). I certainly wouldn't pretend that I've never fallen into one, or both, of those categories myself.

I don't even think editing Documentation is a privilege that should be linked to SO reputation, which is ostensibly a measure of one's problem solving abilities—after all, good technical writing is a skill that does not necessarily bear much correlation to diagnostic skills. I suspect it should perhaps be awarded more manually, based on the quality and detail of one's posts.

Apologies again for this (not very constructive?) rant—I just feel so much effort has already been (and is still to be) expended on this project, and I'm skeptical it's adding any value, that I think it urgently requires deeper thought before it's allowed to progress further.

I'm inclined to suggest that the public beta should be suspended until these problems are addressed.

14
  • 31
    I could not agree more. As much as I would love to encourage and reward people in, say, our tag of mysql, it has been rather awful. There needs to be some tying of tag rep to moderation (including deletes) in my opinion.
    – Drew
    Commented Jul 26, 2016 at 17:28
  • 1
    Also, I have about 5 times invited people who post there in that Docs section to join me in the Campaigns room for consolidating efforts and getting some consistency. And reviewing and approving type of stuff. Only Rick James is there.
    – Drew
    Commented Jul 26, 2016 at 17:36
  • 11
    I totally agree that documentation does not have to do much with SO rating. SO rating is somewhat small indication of anything to be honest (a pity, that), because of the massive amount of users just sitting there and answering same simple/duplicated/easily googled questions. Technical writing skills, structuring material, choosing laconic examples - this is what important when writing documentation. Commented Jul 27, 2016 at 10:42
  • 5
    Very nice post. I see the problem of not enough high quality content, but I'm also worried about the missing structure of the content. It's just all thrown at you. Increasing the expert contributors ratio is one thing but having a better organization of the content is another very important point in my opinion. Commented Jul 28, 2016 at 12:15
  • 9
    I'll add my 2 cents: the name "Documentation" kind of implies a certain level of, well, documentation. I've contributed but, to be honest, between the layout of information and topics, the gamification of the information provided and the general mechanics of it all, documentation it is not. It might be more apt if it were instead named something like "Examples", at least then I have no preconceived notions that I can dive into the C++ topic and start reading specific documentation about the language, syntax, semantics, specifications and/or library abilities..
    – txtechhelp
    Commented Jul 28, 2016 at 23:46
  • 10
    If this is not meant to be actual documentation but just a blob of searchable, community-curated examples, then there are still two problems: - Why is it called Documentation? - How does this add anything that we don't already have with Stack Overflow?
    – Asik
    Commented Jul 29, 2016 at 18:28
  • 6
    Agreed. Documentations need to be accurate and correct, which is not everyone can do. Commented Jul 30, 2016 at 18:00
  • 5
    I agree, public beta should be suspended until these problems are resolved Commented Jul 31, 2016 at 14:15
  • 4
    This, a hundred times this. I won't name the tag but I have visited one and the documentation there is apparently random, poorly written, and just not very good as far as content is concerned. I think the people who are contributing are probably good, knowledgeable people who want to help others; but that isn't enough to write coherent and comprehensive documentation. What we really need is for people to organize teams for each tag and for those teams to define a process to produce documentation. Built-in support from the platform - like is done for voting for mods - would be invaluable.
    – Patrick87
    Commented Aug 1, 2016 at 20:22
  • 2
    I agree with this, plus more. One big problem is that a user will, naturally, focus on the way they do things, which, while not necessarily a bad way to do things may not be the only good way, and could lead to a misleading or restrictive view of a subject. As an example in my field, which is Delphi, a topic of interfaces focusses a lot on TInterfacedObject. Now while that user may always descend from TInterfacedObject, and that may give great benefits to his style of programming, that is not really what interfaces are all about, and that is certainly not the only way to use interfaces.
    – Dsm
    Commented Aug 2, 2016 at 8:26
  • 4
    My main areas are HTML and CSS, and the SO Documentation in those areas is already falling heavily towards a muddle of non-documentation examples plus content plagiarized directly from Mozilla Developer Network, which until now has been the de facto "source" (nb I'm not an MDN editor so don't follow its internals, so don't know if they have trouble with original content vs. plagiarizing from W3). So that's one a problem. Then talking about examples: 1) SO already encourages examples in answers, and 2) how can we make sure examples aren't plagiarized from existing SO answers?
    – henry
    Commented Aug 2, 2016 at 21:16
  • You want to throw out the baby with the bathwater. At least judging from what you wrote, you're not even trying to empathize with the common user who just wants to look at the documentation, you're looking at the whole thing from an incredibly narrow perspective and find that it's not perfect in some way so it should be suspended. This is what people complain about when they say this site has an issue with elitism. Might as well turn off the internet and burn all books by the same logic.
    – Livven
    Commented Aug 3, 2016 at 15:03
  • 2
    I agree with this. Now that I see what it has turned out to be, it's even worse than I feared and warned about. It's a worthless mess. This would be an easy opportunity to grind more reputation in a couple of hours than I've gathered with relevant contributions in the past 5 years, but I don't even want to contribute to this mistake. Keep documentation where it belongs, in the structure it makes most sense in, instead of poorly duplicating it here. Commented Aug 3, 2016 at 18:47
  • 2
    another totally agree comment. I think SE tries to spread itself too thin, rather then sticking to what it does well, question and answer sites.
    – user3956566
    Commented Aug 4, 2016 at 17:35
62

This is pretty great. A tight feedback loop and lots of transparency is always good to see.

I think that the next most important topic after the massive queues is the fairly illogical ordering of the topics on documentation. I can't pick a topic and start learning it, which I think is a great measure of success for the documentation project.

14
  • 1
    Solve with some nice search function.
    – tbodt
    Commented Jul 26, 2016 at 2:44
  • 41
    @tbodt: You seem to miss the point - if you don't know which one you should start from, how would you search?
    – nhahtdh
    Commented Jul 26, 2016 at 3:13
  • 6
    @nhahtdh Yes, I missed the point. I still think search should be improved, though.
    – tbodt
    Commented Jul 26, 2016 at 3:26
  • 1
    Wikipedia has a page that redirects to a random article. It wouldn't be hard to implement something similar, I think. Commented Jul 26, 2016 at 3:43
  • 7
    @dorukayhan You're missing the point. Its not for me to be able to actually pick a random topic. Its for me to be able to choose anything I want and learn it easily. I can't do that now with the current system because there is no organization Commented Jul 26, 2016 at 3:57
  • 28
    It's a cookbook. You don't browse it to learn; you arrive from Google and copy the "example" you want. Commented Jul 26, 2016 at 4:15
  • 5
    "I can't pick a topic and start learning it" Well, no. If you want to learn something properly you need a properly-structured tutorial that you can work through in a linear fashion. Most programming docs do not have that linear structure and so are not a great way to learn a language or framework. Sure, you may need to refer to the docs while working through a tutorial, to get extra details, or to clarify technical issues.
    – PM 2Ring
    Commented Jul 26, 2016 at 12:05
  • 4
    (cont) Hopefully, (once the "teething problems" have been sorted out) the explanations and examples in SO Documentation will act as a useful supplement to the existing docs, but they aren't intended to be a tutorial. OTOH, I guess it would be possible to have a list of "Suggested Reading Order" information for significant topics, but AFAIK that sort of thing is not in the current scope. And while such a thing may be helpful when revising a topic, I don't think it's a good substitute for a properly-structured tutorial when you're trying to learn the material initially.
    – PM 2Ring
    Commented Jul 26, 2016 at 12:05
  • 1
    @PM2Ring this isn't really true.... Java, angular,, mongo, CPP, .net, etc all have learning guides in their docs. We're looking to make the best product out there and I think that includes some structure Commented Jul 26, 2016 at 12:08
  • 4
    Ok. I mostly do Python these days (although I have learned many languages over the decades). The official Python tutorial is excellent (for those who already know how to program), and it is packaged with the rest of the official Python docs, but I tend to think of the tutorial as separate from the Language Reference and Library Reference portions of the docs.
    – PM 2Ring
    Commented Jul 26, 2016 at 12:19
  • 3
    (cont) I see SO Docs as corresponding more to Language and Library docs; I don't see how tutorial material would work well with this format: it's hard to write that sort of material coherently unless it's done by a single author or a small team that know each other well. Crowd-sourcing it sounds like a recipe for an inconsistent mess.
    – PM 2Ring
    Commented Jul 26, 2016 at 12:19
  • 1
    What would be great eventually is to be able to create learning guides that pull good examples from documentation and are linear-based for those just learning a language.
    – Kurtis Beavers Staff
    Commented Jul 26, 2016 at 13:43
  • 1
    @David We shouldn't be aiming to duplicate e.g. the official Java Tutorials. That would be a tremendous waste of effort. I understand why you'd think that, with the feature being named Documentation and all, but it's not intended to replace, well, actual documentation. The example-first/only workflow and tooling clearly points to being a cookbook, and at least that's something that may not exist elsewhere. Commented Jul 26, 2016 at 15:44
  • 1
    @JeffreyBosboom I would like to do both, coming from a search engine with a specific problem but also browsing and learning. So a better ordering of the topics would be highly appreciated. And also for contributing it would be nice to actually get an overview about existing topics more easily. Commented Jul 28, 2016 at 12:27
53

The basic model (rep for example upvotes, approved changes, and upvoting answers citing docs) doesn’t seem terribly broken

It is terribly broken. There are plenty of examples about why it's completely broken. It's not just a matter of threshold configuration or whatever you call it.

Expect more reputation tweaks in the coming weeks

I would expect a complete reworking, not just some tweaks.


From the comments:

That quote is missing the context of "but the limits (like who gets rep for examples, and how much) definitely need work."

It's not a matter of limits, or thresholds. It's basically how the rep model is designed. See the links I provided.

4
  • My comment no longer makes sense in the context of this answers so I'm removing it. The sentiment to it was changes are happening, but it's going to take time, I don't expect it to look the same in a few months as it did at launch.
    – Kurtis Beavers Staff
    Commented Jul 26, 2016 at 13:37
  • I agree changes need to be made. It's been four days since launch, changes are going to continually happen until we get it right. There's no way this community will allow anything less than fair distribution of rep (as we shouldn't.) We had more testing in 24 hours on SO than the entire six months of private beta.
    – Kurtis Beavers Staff
    Commented Jul 26, 2016 at 14:04
  • 11
    @ken2k: The 1 to 1k user thing in hours was fixed when Docs.SO rep was properly capped. That was a bug, and it was fixed within 24 hours of Docs going live. Now, they still get 200 rep per day, so it only takes them 5 days to hit 1k. Commented Jul 26, 2016 at 14:12
  • 2
    @NicolBolas Thanks for pointing this out, it's now indeed 1k every 5 days.
    – ken2k
    Commented Jul 26, 2016 at 14:37
44

Re-posting from Remove or Overhaul Reputation in Documentation


I think that the cheap reputation influx from Documentation may actually have a tangible adverse effect on the participation in the Q&A section of the site, since it disincentivizes users from investing time into actually helping people, when right next door they can make ridiculous amounts of reputation by changing "do not" to "don't", making a Hello World program du jour have proper brackets placement for the language in question and copy-pasting text and source code verbatim from corresponding technology's own documentation.

Call me a rep whore, overcome with green mouth-foaming envy (and you would be absolutely right in this assessment), but I feel utterly disheartened by Docs giving out thousands of rep to users in several days for contributions as valuable and expertise-demanding as I've just described, when achieving comparable results on the Q&A would take months of actual effort and even, occasionally, a shred of expertise.

For me personally this doesn't (yet) kill the motivation to invest time and effort into answering questions on the main site, but it does diminish it quite a bit. And I'm not the only user who feels this way:

Now, I realise that there are low-hanging fruit questions asked every day, which require no effort either. Indeed, my own most upvoted answer is the one stating that private instance variables are accessible from other instances of the same class (some revelatory stuff!). However, most of them get promptly closed as duplicates, since there are only so many basic concepts to clear up in each topic, while others get a few smug comments, a few obvious answers from desperate users with below 1K rep (such as myself) and, crucially, get washed down the ever-flowing stream of history, never to be seen by a potential upvoter again, instead of being nailed to the top of the entire site section, getting constant drops of basically free reputation forever.

So, in short, I don't think that psychological effects of having Docs reputation contribute to the same pool as the Q&A have been great so far and I don't see the situation being improved just by "tweaking" the existing system. I think Documentation should be purely altruistic, for its own quality's sake, or at least that rep pools should be separated.

3
  • 4
    I suppose this is somewhat verifiable. Could track answer upvotes and answers in general and see if the public beta of Docs had an impact.
    – Barry
    Commented Jul 26, 2016 at 23:11
  • 2
    I concur. If anything, please award a flat amount of rep per edit. (i.e. +5 max). I've edited over a 1000 posts on SO and the most I've gotten was +2 rep per post for only the first hundred or so posts I edited. Conversely, you can just ride the rep train by making trivial edits to documentation and earn massive reputation with almost 0 effort. Commented Jul 27, 2016 at 1:13
  • 2
    I definitely agree that the high rep from documentation is a problem. On new q/a sites, it is my understanding that initially reputation is fairly easy to get. Although rep probably needs to be harder to get in documentation, I think rep still needs to be easier to get than on so q/a. I suggest separating rep on the two is and will remain essential. Perhaps cross-site rep should be considered for privileges, though.
    – Bit Chaser
    Commented Jul 29, 2016 at 7:30
33

A lot of contributions I'm seeing are more like, "here's some example code with no explanation of how it works". That's not documentation, that's not even acceptable in an SO answer. When I click on "Proposed changes > Reject", one of the reasons should be "needs more details" or I should be able to link to an existing, more complete entry (if one exists), as is possible with the "dismiss topic" link.

3
  • We should either reject those examples without context in the review process or flag them later for improvement or downvote them. This is a serious problem, but one that is solvable by the current means, I guess. Commented Jul 28, 2016 at 12:30
  • Or one could add a subsequent edit with the required additional information?
    – Toby
    Commented Aug 2, 2016 at 10:36
  • This and: that wouldn't be an acceptable SO answer, but many of the best SO answers include examples. So not only is Documentation heading towards really being Examples, but we already have an existing, stable, well-curated body of examples in the form of SO itself. I hate to say it, but at the time of this writing Documentation is heading towards being a sort of SO knockoff but not as well done.
    – henry
    Commented Aug 2, 2016 at 21:20
15

I think it should also be possible to move topics from one tag to another, if they are requested/created in the wrong tag.

6
  • Possibly. I'd like to wait for some more data here because, if nothing else, the UI is hard to work out in a way that makes any sort of sense... and I sort of expect topics to be migratable between tags, but individual examples are perhaps too granular?
    – Adam Lear StaffMod
    Commented Jul 26, 2016 at 3:18
  • I meant topics... :|
    – tbodt
    Commented Jul 26, 2016 at 3:27
  • @AdamLear You need a better way of correcting tags than having to submit them to a moderator, which is all I could find at present.
    – user207421
    Commented Jul 26, 2016 at 5:53
  • 3
    Regarding moving topics, your deletion and recreation of a very popular topic has led to several complaints on Meta that you may want to respond to: meta.stackoverflow.com/questions/329570/… meta.stackoverflow.com/questions/329596/… meta.stackoverflow.com/questions/329630/…
    – Brad Larson Mod
    Commented Jul 26, 2016 at 14:16
  • 2
    @EJP Did you see the bit about merges and aliases in the question here? That's where we're gonna start. It's a rather destructive operation, so it'll be employee-only to begin with. We'll possibly see about making it available more broadly, but there's of course no ETA at the moment.
    – Adam Lear StaffMod
    Commented Jul 26, 2016 at 14:59
  • @AdamLear I guess a good approach for the UI for such thing would be using the flagging mechanism. You flag for migration, give a reason, the issue is reviewed and if agreed upon the migration is performed. Possibly both, reviewers from the originating tag as well as the target tag have to agree. Commented Jul 28, 2016 at 12:31
14

The biggest problem I see so far with documentation is plagiarism. People are copying and pasting from official documentation without even adding any commentary or usage guidelines, which adds no value and is of questionable legality. Sometimes they'll add a link back, but when 99% of the content on your page is copied from an external source I don't think you can claim fair use.

12

One thing that I faced that might need attention: Creating a topic is too easy. Quite a few times it would suffice to have an example under a - mostly already existing - topic...

For example: Creating an executable jar file in Windows without using IDE

Could we restrict topic creation?

  • some kind of minimal rep?
  • approval required by multiple people (with enough reps)?

This could help in shaping the content, and keep it organized...

11

About the tag part, I don't exactly understand how they work.

See this basic example: Hello World... There is one topic in the .NET tag that gives examples on how to do it in several languages: C#, F#, VB.NET and C++/CLI.

Then there is this topic, in th VB.NET Language tag, that explains how to build Hello World in VB.NET.

There is no link from one topic to another, but clearly there should be, as it is the exact same thing they are trying to achieve...

I think that there should be a difference between an actual programming language and a library that can be used on many languages.

  • An actual language, you should be able to write example out of it, and if your topic is linked to a library, you can tag this library (like the .NET Framework)
  • A library, you should not be able to write an example only with this library tag (no one can code only in the .NET, you still need a language...). A library is tagged in the examples. And then is viewable in the all topics but specifying somehow that this is not a language...)

I don't know if this looks interesting to anyone, but I felt like sharing this, as it might be a start for writing a documentation library based on language examples...

4
  • 3
    Similar issues arise with frameworks available on both iOS and OS X, which are used in Objectvie-C and Swift languages. There is not a clear way to categorize topics and we can expect duplicates.
    – Tricertops
    Commented Jul 26, 2016 at 9:05
  • 1
    @Tricertops That's what I'm afraid of... Without a good categorization and organisation, this whole documentation project might become unusable because unclear... Commented Jul 26, 2016 at 9:39
  • 1
    Good multi-language support would fix this problem. ...along with a massive clean up effort.
    – RubberDuck
    Commented Jul 28, 2016 at 21:43
  • 1
    Having more than a single tag per topic would be nice. Having subtags would be nice. ... As in the Q&A part of SO. Commented Jul 29, 2016 at 9:30
10

Thanks for the update! Weekly updates are a great idea, especially in the beginning now.

Now a little feedback on your planned improvements:

The review queue for proposed changes

I perhaps would not put everything in this queue. The tags where there are enough contributors, especially with enough people having the necessary domain knowledge, work fine currently.

It is easy, as someone with not so much domain knowledge about a specific topic to accidentally approve something factually wrong. Currently, I'm regularly checking through the list of already approved proposals to spot whether the approvals were fine... They really aren't always. It is already bad enough [I've had everyday seen 1-2 factually wrong proposals which got approved]. I cannot imagine a general purpose review queue improving this situation. See also: What can we do about robo-reviewing on documentation?

Thus, I'd suggest that the review queue to be used mainly for low-frequency tags where the time until approval is exceeding 12-24 hours. As it is really valuable to have the proposals reviewed by domain experts. Just in case there are none, it must go through the generic queue.

Alternatively, you can have the queue by default only for low-frequency tags and allow people to opt-in for other tags (if they are knowledgeable there).

Perhaps, there are better ways than I proposed, the main point is:

  • Reviewing well-frequented tags [where enough reviewers exist] should definitely not be a low hanging fruit.

Apart from that: Thanks for including "Edit & Approve". Editing should though not immediately be approved, else it is an easy way to game the system and introduce yourself factually wrong information without further review. (see also Improving requested edits in Stack Overflow Documentation)

Tag merges

They are nice to have, and I can just think of a few issues in case a tag needs to be split up (some topics belong more to one tag, and some more to the other one), or only parts of a tag needing to be split out. This should be taken into consideration.

4
  • 6
    Been wondering about review queue myself. It's relatively easy to review Q/A in majority of cases without knowing anything about a language , it's usually about structure of the post itself. Docs examples are the opposite
    – charlietfl
    Commented Jul 26, 2016 at 2:57
  • Exactly my point. And thus it should be tried to favor those people for reviewing who can really verify it instead of the opposite [=> general public of people with 2k+ rep [or whatever the limits will be on that queue]].
    – bwoebi
    Commented Jul 26, 2016 at 2:59
  • However...click on image and note there is a tag filter showing
    – charlietfl
    Commented Jul 26, 2016 at 3:06
  • 1
    @charlietfl that's a filter like the normal suggested edits queue also has one. The filter is off by default, I suppose, though.
    – bwoebi
    Commented Jul 26, 2016 at 3:08
4

I am not sure if I understand the overall reason for the Documentation site; I mean, it seems very structured and formatted as if there is only one way to write documentation.

I understand the need for more examples, but a lot of good documentation is prose.

Some specific observations:

  • Why, after I click on the DocumentationBeta link, and then click on Questions, does it take me to the recents tab? I thought it would take me the Home tab.

    • From the StackOverflow home page, click on DocumentationBeta
    • Next, Click on Questions (on the top of the site, to the left of the Jobs link).
    • Result: You are taken to the main StackOverflow page, with the "recents" tab selected.
    • Expected: You are taken to the main StackOverflow page, with the "home" tab selected.
  • Is there a way to allow monospaced text, without the grey background? This is useful in tables when defining parameter values.

  • Speaking of the Parameters section, why can't I write normal text in it? Sometimes, a parameter requires a footnote or other explanation that is out of place in a table. However, when I tried to do this it gave me a "can only contain two column table" error. Why this specific restriction?

5
  • It may help if you think of Docs as a cookbook, not a tutorial or reference manual; it doesn't have to replace the latter. The devs have said they strongly considered naming the feature Examples, and the private beta help said ideal examples were copy-pasteable (though they've since backed away from that). To put it another way, they committed to doing one thing (examples) and doing it well. See also Jon Ericson's answer, which is the best quasi-official explanation so far. Commented Aug 1, 2016 at 5:07
  • What do you mean by "click on Home"? Or "Home tab". I might be missing something here, but I don't see those terms anywhere on the page?
    – Adam Lear StaffMod
    Commented Aug 1, 2016 at 7:21
  • Sorry Adam, it was a pre-coffee posting. I mean the Questions link. Commented Aug 1, 2016 at 7:27
  • 1
    @JeffreyBosboom "It may help if you think of Docs as a cookbook, not a tutorial or reference manual": herein lies the main problem with SO Docs. no one knows what it's supposed to be. You think it's cookbooks. Others think it's reference material. Yet some others think that "Documentation" implies, well, actual documentation. So what it it, really?
    – Mamut
    Commented Aug 1, 2016 at 8:01
  • "good documentation is prose". Quite. I think a better name for now would be Spaghetti: as in that-which-one-throws-at-the-wall-to-see-if-it-sticks.
    – Benjol
    Commented Aug 2, 2016 at 5:31
1

With regard to the problem of too many examples per topic - I think this comes back to the structure of documentation and the two ways that this allows it to be used.


  • As a book - i.e. a topic is a chapter.

Thus a "String Manipulation" topic will have lots of examples for how to perform the multiple ways of manipulating strings (creation, concatenation, copying, searching, splitting, length-counting, character and sub-string replacement, etc) in different contexts and use-cases (At definition time, statically allocated strings, dynamically allocated strings, best practice vs low-memory environments, etc).

  • As a reference - i.e. a topic is based around one single facet.

Thus a single topic could be "String Concatenation" and then provide documentation for the basic string concatenation function of a language covering only the common, basic, use-cases. Other string concatenation functions or use cases would be covered in more specifically named topics.


Here is the issue. Documentation is structured for the latter (a topic name, a small number examples, a syntax section [itself implying application to a single code operation], and the remarks section being towards the end rather than the start), but is used for the former (titles over concepts rather than specifics, remarks section used as topic descriptions rather than short, well, remarks, syntax sections being over-run with attempts to document syntax for all the items documented within a chapter topic).

Personally I feel that the current usage we see is actually producing a more helpful Documentation than a reference style. But it does not suit the current structure. The resulting conclusion being that the structure of Documentation needs rejigging to better match the reality of its use.

Not the answer you're looking for? Browse other questions tagged .