334

Early on, I made some small change to some small part of PHP Documentation and occasionally see a single rep point pop up, but I NEVER find myself visiting it for ANY reason now.

Matteo Italia's well upvoted answer to a question about improving Documentation said "...just stop caring." and "don't waste your time trying to fix it, just let it die"

If this is the general consensus of SO users, has the time come to pull the proverbial plug on it?

53
  • 117
    SE's response to pretty much all of the criticism of Documentation has been to ignore it, rather than fixing any of the problems. They've pretty strongly indicated that they're not going to pull it because they think it's successful, despite most of the site regulars disagreeing. Presumably this is because it's getting lots of views, even though subject matter experts feel it doesn't have quality content, and that's enough for SE.
    – Servy
    Commented Apr 17, 2017 at 20:27
  • 43
    @Servy We haven't done a great job communicating here, but the internal perception is emphatically not that it's a huge success and everything is great. We definitely realize it needs some improving and we should come out with some updates on the roadmap soon.
    – David Fullerton Mod
    Commented Apr 17, 2017 at 20:32
  • 26
    @Servy: "SE's response to pretty much all of the criticism of Documentation has been to ignore it" Not all of their response to criticism has been to ignore it. Sometimes they acknowledge it but then say "just wait, it'll work out." Commented Apr 17, 2017 at 20:32
  • 91
    @DavidFullerton: "we definitely realize it needs some improving" My concern is really that the SE staff has not demonstrated an understanding of the problems and flaws of the Docs.SO system, both conceptually and implementation-wise. Whenever these flaws are enumerated, we're generally told that we don't understand the system, or that people will figure out how to use it better, or something dismissive like that. You cannot improve a system if you cannot acknowledge what is actually wrong with it. Not merely that it isn't working (anyone can see that), but why it isn't working. Commented Apr 17, 2017 at 20:38
  • 16
    @NicolBolas Well, you can argue that we're incompetent, but at least we're not ignoring the fact that it's not working right now and pretending everything is okay :) Throwing everything away and starting over with a different format (as you've suggested) is always a possibility, but we'd really like to fix what we've got if at all possible.
    – David Fullerton Mod
    Commented Apr 17, 2017 at 20:47
  • 31
    I watched Documentation from its proposal, through its various stages, with interest. Not because I understood it, I thought from the beginning that it seemed to be an ill-defined solution in search of a convincing problem. I was honestly hoping to be proven wrong, and I'm far from happy to be – so far as I can currently see – right. Commented Apr 17, 2017 at 23:18
  • 17
    @Jaydles But you never acknowledge the criticisms that the project is poorly motivated or that the experts won't engage or that the site will silt up with rubbish, all of which you were told a year ago. SO continues to think of it as something that just needs more tweaking. The problems are far deeper than that, and they are mostly intrinsic.
    – user207421
    Commented Apr 18, 2017 at 3:44
  • 16
    The only reason I remember Documentation is still a thing are the occasional notifications that somebody has proposed a change to a topic I contributed to ages ago and have long given up on. They would have been reviewed by the time I see the notifications, and I have no way of disassociating from the topic so I stop seeing them in the first place, so I don't care about them anymore.
    – BoltClock Mod
    Commented Apr 18, 2017 at 4:48
  • 18
    Haven't we had this post several times already? Yes it is a failure, let it die in peace.
    – Lundin
    Commented Apr 18, 2017 at 8:03
  • 26
    I can;t claim any such hatred, but it's been my observation that the ones with most animosity towards Documentation tend to be those who work hard to establish and maintain their reputation. The award of rep for contributions to Docs appears to be incommensurate with the amount of effort expended, and no penalty appears to be imposed for inaccurate or misleading contributions.
    – holdenweb
    Commented Apr 18, 2017 at 10:01
  • 9
    My own experience is that topics tend to become "full" of drivel that runs the risk of misleading those wishing to learn. Further contribution attempts are then rejected with cries of "no room!"
    – holdenweb
    Commented Apr 18, 2017 at 10:04
  • 49
    @SGR: "What I've never understood since the start of Documentation is the sheer unbridled hatred some users have for it on this site." Docs.SO is 1) a poorly conceived feature, 2) many people told them it was poorly conceived, 3) is implemented badly, 4) we told them it was implemented badly in the beta, 5) yet despite all of this, they keep moving forward with it. This kind of pig-headed determination to cling to bad ideas tends to engender hatred from the people those ideas are being foisted on. Commented Apr 18, 2017 at 13:38
  • 24
    @Nathan: Even if you ignore the big "Documentation Beta" button at the top, it affects rep gaining significantly, leading to many people having Q&A privileges that they didn't effectively earn. People are reviewing edits to SO who haven't the slightest idea how Q&A works. I would consider that being "foisted on" everyone. Commented Apr 19, 2017 at 2:39
  • 31
    When you come to realise that even w3schools is a better documentation portal then SO.Docs, something is seriously wrong with the approach.
    – user692942
    Commented Apr 19, 2017 at 9:31
  • 12
    I mean, just ask yourself: stackoverflow.com/documentation, or php.net/docs.php? stackoverflow.com/documentation, or msdn.microsoft.com/en-us/default.aspx? stackoverflow.com/documentation, or developer.mozilla.org/en-US/docs/Web/JavaScript? stackoverflow.com/documentation, or api.jquery.com? stackoverflow.com/documentation, or google.com? You get the drift. The choices should've been clear before you guys dedicated development resources to it. My browser's address bar search > SO docs search box, in all cases.
    – Jason C
    Commented Apr 20, 2017 at 2:14

13 Answers 13

186

How do you determine if it has failed? There are so many standards by which you could measure the current functionality of Docs.SO as a system.

So let us look at the foundation document for Docs.SO: Warlords of Documentation. It enumerates four broad problems with documentation, so let's see if we've succeeded at any of them:

  1. Documentation is often an afterthought, obviously done just to say that it exists, with little concern for its quality. We've all seen plenty of this in our time. We can bring a focus on quality, peer review, and "actually solves real problems for real developers"-ness to documentation that would be very welcome.

I would say that, objectively speaking, quality on Docs.SO is... scattershot. I'm sure there are some good examples out there, and even whole topics that are decent. But as a whole, I'd say that whatever quality is there is drowning under a tide of (at best) mediocrity. And plagiarism.

Peer review on Docs.SO? Complete and utter failure. The fact that it takes just any 4 individuals to allow anything on the site, even when none of them has the specific domain knowledge or is otherwise a suitable reviewer, shows the failure of this system. This is a non-trivial part of the lack of quality of Docs.SO.

As for solving "real problems for real developers"... I'd say that Docs.SO by and large doesn't do that. Topics tend to be more of the "basic documentation" variety rather than anything problem or task focused. As such, examples more often than not demonstrate how a tool works, not how you're supposed to use it to accomplish something useful.

So overall, #1 is a bust.

  1. Often documentation is lacking in examples, or the examples are trivial and don't demonstrate typical use. Because Stack Overflow benefits from constant feedback from developers writing real-world code, we could greatly improve the quantity and quality of examples. Let's be honest: finding examples is already a common use-case for Stack Overflow; we might as well embrace it.

Docs.SO is example-focused documentation. In a technical sense; the meat of topics are called examples.

However, most examples on Docs.SO are not "real-world code". They're artificial, used to explain a specific point. They show off how to use a method or a class or something of that sort. But very little of it is something from "real-world code".

So #2 doesn't seem to be any good either.

  1. Much of the documentation out there is tied to release cycles, and thus infrequently updated and rarely "complete". The community-contributed and edited nature of Stack Overflow would be an immediate improvement.

It's hard to evaluate how well Docs.SO handles being out of date. It does however acknowledge that versions are things that exists, and provides tools that allow users to add version-specific information.

However, considering the low participation in Docs.SO, it is highly unlikely that, as new releases of software are made, Docs.SO will be updated in anything like a comprehensive fashion. So as time passes, Docs.SO will become increasingly out-of-date.

This isn't a structural problem so much as a participation problem. If Docs.SO were actively maintained by a set of energetic and skilled hands, this would probably work out.

  1. A lot of documentation descends from Javadocs, and while it was better than nothing in '96, it's hard to call framesets with unshareable URLs "good" in 2015. We can focus on creating the best UX for creating and serving documentation on the modern web.

This is probably more subjective than most, but quite frankly, I'll take JavaDoc organization over Docs.SO any day of the week. Oh sure, you can link directly to an example, or to the syntax or remarks section. You can even link directly to a specific version, to ensure that you're showing off something that won't change.

But if you need to know what a function does, it's still a lot easier to find a JavaDoc page on it than the appropriate Docs.SO article. Docs.SO's search is horrible, which makes finding specific information essentially impossible. You can't just type in a class/function name and expect to find something on it.

JavaDoc may have terrible HTML, but if you know the class and member function that's causing a problem, you can at least find their crappy documentation about it. We can't even get that far.

So, is Docs.SO a failure? I would say that there's good evidence that it has yet to accomplish any of the stated goals of the project.

20
  • 7
    Based on what I think Docs is supposed to be (examples, not Javadoc), a better comparison would be the Java Tutorials, which can be navigated and linked into just fine. Here's the page on Swing combo boxes, for example. I got there by following the hierarchy, but I could also have Ctrl-F'd on the Really Big Index, and of course Google indexes it. So point 4 can't really be improved upon, anyway. Commented Apr 17, 2017 at 22:43
  • 21
    "The fact that it takes just 4 idiots to allow anything on the site shows the failure of this system" - this has been a large part of my problem with SO (especially but not only the review part) for years... Commented Apr 17, 2017 at 22:48
  • 67
    @Dukeling: But Q&A reviewing at least requires 2000 rep. Anyone who just registered is able to approve Docs.SO changes. Commented Apr 17, 2017 at 22:50
  • 26
    #1: The entire existence of Docs.so is an afterthought, by definition. It is intrinsically impossible for StackOverflow to solve that problem. #2 'Lacking in examples' is not a sufficient motivation for deciding that there should be another whole system that is example-based. #3 'Tied to release cycles' is exactly what documentation should be. No real problem is being addressed here. #4 Yet another straw man. Javadoc does not consist of 'unshareable links', and there are many thousands of SO posts that prove it. Again no real problem is being addressed here. These motivations are all void.
    – user207421
    Commented Apr 17, 2017 at 23:07
  • 14
    'Spurious' would a better word. The project has failed for reasons which were comprehensively predicted a year ago: it is poorly motivated; the experts won't engage; and it will silt up with rubbish, which is exactly what has happened.
    – user207421
    Commented Apr 18, 2017 at 3:41
  • 12
    @Jeffrey Bosboom: Of course, thanks to the misleading name "Documentation" many people think it's a place to find API documentation or step-by-step tutorials anyways, so the comparison to Javadoc probably isn't that far off. Just look at all the other tags whose "Documentation" consists primarily of content copied from official manuals.
    – BoltClock Mod
    Commented Apr 18, 2017 at 4:50
  • 4
    But you can link directly to any section in JavaDoc... they've all got anchors etc.
    – OrangeDog
    Commented Apr 18, 2017 at 16:12
  • 3
    The fact that any post with this in it: "The fact that it takes just 4 idiots to allow anything on the site shows the failure of this system" gets so many upvotes reveals why SO has degenerated into a horrible place. Why do we continue using this thing? We could make something that actually was focused on knowledge that required civil exchange. Maybe we should do that?
    – codewise
    Commented Apr 18, 2017 at 21:49
  • 11
    @codewise: "We could make something that actually was focused on knowledge that required civil exchange." Would you feel better if I called them "intellectually challenged?" You're ignoring the main point (ie: low quality of information due to people approving changes who have no idea what they're doing) to dither over a matter of what we call such people. Commented Apr 18, 2017 at 23:34
  • 6
    @NicolBolas The tone is what scares learners away from this SO. I agree that documentation should be peer-reviewed and perhaps the requirements should be more stringent. But name calling is essentially a form of attack. It does not help anyone.
    – codewise
    Commented Apr 19, 2017 at 5:14
  • 10
    "4 people who lack specific domain knowledge" @codewise
    – Cody Gray Mod
    Commented Apr 19, 2017 at 10:11
  • 4
    And the reason a project doesn't have good documentation is probably because the community that uses it is not motivated enough to add it so I cannot see why they would be anymore motivated to add it to SO if they won't do it within the project itself.
    – br3w5
    Commented Apr 19, 2017 at 11:05
  • 9
    @CodyGray: The problem with "people who lack specific domain knowledge" is that such people are not the problem. The problem are people who lack specific domain knowledge, yet are arrogant enough to believe that they actually know enough about that domain to review the work of others. Ignorance alone is not the problem, so the description is not accurate. Commented Apr 19, 2017 at 16:32
  • 22
    At this point we should really just rename Documentation to Code or Code Snippets and remove all the 'doc' verbiage and just focus solely on examples of how to do certain popular things in a given language. Oh, and for starters, users should require at least a thousand rep to suggest an example, and have a bronze badge in the tag in order to review suggestions, with gold badgers obviously able to unilaterally approve/reject suggestions and make unilateral edits.
    – TylerH
    Commented Apr 19, 2017 at 18:27
  • 2
    @TylerH "examples of how to do certain popular things in a given language" So something like Rosetta Code, but 'transposed', such that examples are listed by language/library and not by task? Commented Apr 21, 2017 at 3:25
102

In my humble opinion, the answer is unequivocally yes. It is a failed experiment.

I thought Documentation was a solution in search of a problem from when it was first announced. But I didn't complain on Meta, I simply decided to wait and see if it would actually work. I think it's time to admit it doesn't.

I've used Stack Overflow regularly since 2009, I'm in the top 0.7% of reputation or something like that, and I'm ostensibly a subject matter expert.

I am also amazingly passionate about software documentation, and I work hard to ensure that APIs I create and projects I maintain are well documented. Fifteen-ish years ago I even worked professionally as a technical writer for a while. So this kind of thing would seem to be right up my alley.

But I have literally never used the Documentation feature.

Not only have I never contributed to Documentation, I have never even used it. An internet search for documentation -- about literally anything I have ever searched for -- has never turned up Documentation as one of the top hits. stackoverflow.com comes up all the time. Other curated document sets on GitHub, readthedocs.io, or other sites come up all the time. Stack Overflow Documentation? Nope.

Every time I start to wonder if Documentation is worth my time, I see posts about what a tragic mess it is. If I'm going to invest in documenting a project, I'll contribute to the official documentation, thank you very much.

And if I'm consuming documentation, I'm going to look for high-quality, authoritative material. And I hate to say it, but Stack Overflow Documentation isn't going to be where I find it.

6
  • 52
    I think they are deliberately keeping it away from the search engines, since it is supposedly still in beta state. If Documentation were to pop up in search engine results, it would probably mean that SO as whole would lose a lot of credibility and get lower rank.
    – Lundin
    Commented Apr 18, 2017 at 8:06
  • 24
    Like you, I am something of an expert in certain technologies, and passionate about both documentation and accuracy. Unlike you, I actually did try to use Documentation. It was an unabashed failure. The misinformation, plagiarized content, and sheer garbage poured in (and was approved by others who were similarly ill-informed) faster than I could ever hope to deal with it. The UI was actively hostile in many ways. The project's purpose was ill-conceived and remains ill-defined, define earnest pleas—is it a "cookbook" of examples or API "documentation" in the traditional sense? No one knows.
    – Cody Gray Mod
    Commented Apr 19, 2017 at 10:14
  • SO documentation is hosted under stackoverflow.com URN, therefore, when more relevant resources exist on SO main part, they will be displayed first. Google likes displaying one result per site, and indeed documentation won't show due to many reasons, probably including tailored parser for SO Q&A on Google side.
    – Vesper
    Commented Apr 19, 2017 at 13:00
  • So long story short: the experiment failed because you have never used it. From my perspective, Chinese is a failed experiment.
    – Sherlock
    Commented Apr 24, 2017 at 9:47
  • 4
    @Sherlock: That's not quite a fair characterization of my post. A better metaphor would be: If I, a person who is deeply interested in communicating with Chinese people, have never found any use for the Chinese language when actually communicating with Chinese people, then the Chinese language could be considered a failed experiment. SO Documentation is clearly in a very different situation than the Chinese language! Commented Apr 24, 2017 at 16:26
  • 1
    My point was that your answer is an argumentum ad verecundiam. It lacks analysis of the actual experiment (see Nicol Bolas' answer).
    – Sherlock
    Commented Apr 26, 2017 at 9:23
98

I haven't been closely involved with the Documentation project. I've been watching it from afar with some interest, and I pitched in here on meta for a few weeks after the public beta rolled out, but outside of a few really frustrating discussions about reputation I haven't really been a part of the design or direction. So... I'm not particularly qualified to talk about it. But hey, that's never stopped me before... And there's something really critical missing from this discussion.

See, I've been watching this thread for a couple days now, and something's really bugging me about the responses. Specifically, the responses from employees of Stack Overflow The Company. They're all... kinda myopic.

I don't mean that as a put-down; the executives, project manager and community manager involved here are all doing their best to provide accurate, specific context for how we're viewing this project, what we're trying to achieve and how we're trying to approach that. And it's all really good information - if you already know where we're coming from.

But, you probably don't know that. There's, what, a year and a half worth of really dense meta posts, blog posts, answers, comments, speculation and discussion surrounding this project now, and nobody got time for that. Also... Even if you read all of it, you kinda had to read between the lines to figure out what was really going on.

So... I'm gonna try to provide some context, and then I'll try to answer your question.

Too long, won't read

If you're already bored, stop reading this answer and go read this one. Come back only if you don't understand that answer... Or don't believe it.

What follows is my personal take on where this whole "Documentation" thing comes from, and where it's going. As is my tradition, it is long, rambling, and full of digressions... You have been warned.

The chaotic money machine

Years ago, long-time SO/SE member jmort253 pointed me to this entertaining talk from Mailchimp founder Ben Chestnut. One part in particular stuck with me, since it describes one of the primary challenges facing both Stack Overflow the Q&A site and Stack Overflow the company. It's a fun video, but for the sake of those who don't have 40 minutes to burn I'm gonna quote the relevant bit from a transcript I found on Chris Barber's blog:

An entrepreneur has an idea usually. He wants to start a company. A business is like the steam machine like you don’t know how this works. You start up a business. You're like, "Oh, if I tweak this knob, I think money comes out."

If I adjust the screw or like maybe make the pulley or something tighter like more money will come out. That's the first couple of years and then after a little while, you're like, "Wait a minute. Wait a minute. Wait a minute." Two knobs. What happens then like, "Holy shit. Wow. ..." Then, you're like, "Wow, man. I kind of get this stuff. This is kind of cool." Now, I'm going to start thinking big like Richard Branson big. You start learning about key performance indicators like my KPIs are all like knobs. I'm like him but knobs on top of knobs and I've got knobs down here and then like bam.

[...]

Things go wrong when that original entrepreneur, the creative guy, says,

"You know what? I deserve a break. I'm going to delegate now. The business is running itself. I can sort of like step back a little bit. Hands off."

[...]

The thing is, your managers back at the office like, "What do I do with this thing?" He didn’t leave a manual. I don’t operate this stuff. I just protect so I'm going to hire robots and they're going to guard it. That's what I do.

[...]

Before you know it, your whole company is thinking like managers. You're not all managers but you're thinking like managers. You're defending the money machine that you made 10 years ago. No one's making new machines. No one's looking to improve it. You're just defending, defending against competition, whatever. You're just in defense mode and even worse, the creative people at the bottom, they're like, "Wow! The only way to move up in this company is to become a manager or think like a manager."

We pretty much had the whole "programmer Q&A" thing ticking along 8 years ago. There was still a lot of optimization to be done, a lot of knob-turning and such, figuring out how to scale it up and make it run smoothly... But the basic engine was running; it worked, it made money, it generated a repository of knowledge. Not very much knowledge, but... Enough to demonstrate the concept. Six years ago, that pile of knowledge was a lot bigger... And we were starting to see that defensive thinking set in, both in the folks here on meta and within the company. Today, we have millions of users and hundreds of employees who all depend on this machine running constantly, consistently, every day 24/7. Changing anything that might threaten that operation requires getting past row after row of guards, on the site, here on meta, and within the company itself. We're all in "defense mode":

diagram from that video showing a machine being protected by a silly amount of auxiliary stuff

The changes that do get made have to work around or compensate for deficiencies in all of the stuff that doesn't change; the end result of this is a huge, impossibly complicated system that few fully understand and where no one can predict what changing it will actually do.

At some level, anyone who's been around more than 5 minutes knows that this is problematic, that we're ossifying, burning out, expending all of our energy just to maintain position...

Pasta on the wall

More than ever, we need to be trying new things, looking for new solutions to old problems and unmet challenges to rise to. At the same time, it has never been harder to actually try anything. The crushing presence of an immense critical audience awaits; one step more and the guard dogs will be snapping at your femoral artery.

You don't have to look hard to see the signs of this: the unchanging "Ask Question" page, ancient editor, total lack of any encouragement for experienced members to maintain aging information... Anything that might cause the machine to stall becomes untouchable. We all know it's a problem even as we're all part of the problem!

So a few years back, some folks within the company decided to push in a different direction: if we can't experiment on the main Q&A system, let's build new systems, and test ideas that way. Let's throw a whole bunch of pasta against the wall & see what sticks. Will it be mentoring? Videos? Teams? Stories? International sites? Documentation? Dancing?

And so the call went out: come up with an idea, convince your co-workers to try it, convince the exec team to try it, then convince meta. Then build it & see what happens...

A guy in a park, dancing in front of a tablet to log in. Sometimes, pasta is messy.

Trust us: we think we don't know what we're doing

If all of this sounds kinda crazy... Maybe even a little bit irresponsible... You got the right idea. We're in defense-mode for a reason after all; a lot of the ideas folks have been proposing are kinda dangerous; if we tried 'em on Q&A, they probably would break things badly, so it shouldn't surprise anyone to see that when we try 'em in Documentation they are breaking. The big difference is... We can still fix them. And we can learn something from the experience without destroying 9 years of work in the process.

Stack Overflow Q&A today looks nothing like it did in the early days. The most popular questions on the site in early '09 were those about cartoons, hidden features, jokes, and questions about Stack Overflow the site itself! It's pretty safe to say that if you took a regular SO user today, transported them back in time and dropped them into the site during its first year... They would be horrified by the lack of focus and utter chaos that reigned supreme during those days... Then they would be delighted by the lack of restrictions... Then they would be dismayed by the lack of useful information.

We got the site we have today because we tried a bunch of stuff early on, and learned from our mistakes. We need to be able to keep doing that.

Finally, the answer to your question: has Documentation failed?

Yes! Several times, in fact. I'd give you a laundry-list, but as I said at the outset there are a bunch of folks way more qualified to talk about this than me; instead, I'll point out perhaps the most blatant example:

Reputation was utterly broken at the launch of the beta. No sugar-coating it; the rep system in place at the start of the public beta was completely unworkable, and fell apart within days. What'd we learn from this? Exactly what we'd suspected since literally the private beta of Stack Overflow back in '08: that awarding reputation for editing is really, really problematic. Which is why we never ventured to touch that hornets' nest in Q&A; it would have broken everything. With Docs, we had the chance to test it, and fix it. (I'm not at all convinced that we have it right just yet, but we have made a LOT of progress)

If you're patting yourself on the back right now for knowing ahead of time that the rep system was broken... So am I; it makes it hard to type. Also, it's a bogus attitude - neither of us knew it would fail, we just made an educated guess - now we have actual experimental evidence, copious detail on exactly how it failed. If we ever want to make concrete improvements, the latter is infinitely more useful than the former. I'm sure Jon and Kevin and Vasudha have a long list of other failures - and what we've learned from them, and what we're gonna do to avoid them in the future.

In closing

It took at least a couple of years before the "best of" Stack Overflow didn't look like a joke. An awful lot of folks gave up on it before then, either because they didn't have the time to stick it out or because it was turning into something completely different from the site they thought they'd signed up for. If we stick with Documentation that long, I fully expect to see the same thing happen... But even without that rosy future, I'm excited that we're finally making big mistakes and learning from them again, turning knobs and seeing what happens. And even if you don't ever intend to use Documentation, you should be excited about that too. Everyone needs a little bit of chaos...

56
  • 12
    so that is a clear path for cash-starved start-ups. But when you have cash, you can afford to do it the regular time-proven way. Define your concept, the value proposition, the target market, validate the product/market fit through focus groups or else, and so on, and so forth. I cant sugar coat it either. SO Docs did not receive the level of study that is necessary to ensure its success. It really looks like the idea of one guy, and lets see what gives. There is nothing to be proud of here. When you roll out a product to millions of people, you want to have done your homework before. Commented Apr 19, 2017 at 22:20
  • 9
    Funny thing, @user3743222... I'm pretty sure Docs has been focus-group-tested more than any other part of Stack Overflow, ever. If memory serves, focus groups delayed the original launch by months. Focus groups have their place, but they don't tell you how something is gonna be used by a few hundred people all hammering on it at once. Focus groups don't tell you that your incentive system is gonna fall apart at scale - they may even encourage poor incentives. Surveys & little test groups & interviews all have their place, but they don't magically replace people hammering away in earnest.
    – Shog9
    Commented Apr 19, 2017 at 22:29
  • 14
    Don't mean to be unkind here, @user3743222, but "get Marketing involved" isn't a process any more than "get Design involved" is a process. Marketing and design (and testing, and research, and SRE and legal and ...) aren't magic fairy dust that you sprinkle onto a project to make it work; they're parts of a team that can assist if they're involved from the start. FWIW, we've only had a marketing dept for SO for maybe 3 years, and Documentation and Developer Story are probably the projects they've been most involved in; they did good work, but... Not magic.
    – Shog9
    Commented Apr 19, 2017 at 22:42
  • 8
    @Shog9: "The most popular questions on the site in early '09" I do not think that the most popular questions is a useful metric to determine if the site was useful or not. A better metric would be to look at the influx of new questions (at the time) and ask how many of them would be considered on-topic today. And whether they were getting good answers. Because if that is working (and it was), you simply need to eliminate the non-working parts. Right now, you can't point at anything in Docs.SO and claim that it's working as it should. Commented Apr 19, 2017 at 23:29
  • 10
    @Shog9: "Stack Overflow was still completely worthless as a tool for doing my job and would continue to be for a fair bit longer." Why was that the case? Were questions about useful stuff being ignored, or was it difficult to find questions about useful stuff? Was it the lack of a categorization mechanism or something else that made it hard to find good questions? Because the list of popular questions is essentially irrelevant; nobody uses that to find questions to learn from or answer. Commented Apr 19, 2017 at 23:37
  • 14
    Well now... If only this answer had been part of the original proposal. From the beginning I think a lot of people have been very confused about the purpose of Documentation above and beyond Stack Overflow Q&A. If I had known what I now know from reading this answer, I would have been much more supportive throughout. I get the idea that you need to be able to fail horribly to make a great product.
    – user4639281
    Commented Apr 19, 2017 at 23:50
  • 7
    My point here is, maybe you are imposing this problem on yourself. SO Q&A is perfectly fine without attributing reputation for editing. Is there anybody complaining about that? Why try to fix something that has no problem? Maybe SO Docs needs no reputation system? The bias that comes from the experience of Q&A where the reputation did work (eventually, according to what you say) is that there needs to be a reputation system too for SO Docs. Maybe thats not the case. In any case, the first thing is to have a conceptually well-defined SO Docs, otherwise it is so much harder to reason about. Commented Apr 20, 2017 at 0:17
  • 7
    So, I've seen a couple of employee answers say something like "Q&A isn't working (people won't self-answer, newbies find the rules inscrutable, ...), so we'll try something new", and this post seems to say Q&A is ossified, implying that the perennially-promised "quality project" that would improve question quality through better asker guidance or whatever won't happen. That's disappointing, but understandable. But Docs is moving slowly enough that I feel like it's already ossified too (especially because you built your own sort-of-wiki instead of using an existing one). What did you gain? Commented Apr 20, 2017 at 5:09
  • 8
    “come up with an idea, convince your co-workers to try it, convince the exec team to try it, then convince meta – I don’t think meta was convinced when the Docs project was originally announced. There were a lot of concerns and confusion about its purpose (it following the original Q/A idea of a knowledge database was never mentioned officially btw.). But the staff responses were mostly “just let us do our thing and you will see”, as if they always knew better. But even with the first beta they failed to answer most concerns; and meta kept staying largely unconvinced.
    – poke
    Commented Apr 20, 2017 at 14:50
  • 10
    This answer sums up pretty well why I've always felt Documentation deserved defending against the "Y U DO NEW THINGS! Y U NOT FOCUS ON MAIN THING!!!!!" crowd even if it might ultimately end up failing.
    – Pekka
    Commented Apr 20, 2017 at 14:58
  • 8
    “I'm pretty sure Docs has been focus-group-tested more than any other part of Stack Overflow, ever.” – That does not really explain why Docs – to me – has the worst kind of UX ever. To this date, I literally cannot navigate that thing properly, or figure out how all those different edit histories interlock with each other. It overall feels rather counter intuitive for its purpose (allowing easy edits, and proper reviewing of content changes).
    – poke
    Commented Apr 20, 2017 at 15:01
  • 6
    Sorta, @Caleb. Stack Overflow has generated an immense volume of useful, practical solutions to programming problems, and thousands of new questions are answered every day - in that sense it is a success. That success is no small feat! At the same time... An awful lot of practical questions end up never being asked here, or remain unanswered; the depth of knowledge for small tags can be pitiful. There is a growing volume of noise that makes it hard to find useful information. There's a perception (somewhat exaggerated) that crowdsourced debugging is primary purpose. These are failures.
    – Shog9
    Commented Apr 20, 2017 at 18:26
  • 6
    The danger here is that we've come to fear failure so much that we resist even trying to fix the problems lest we make them worse or break something else. That we become so protective of the areas in which we've succeeded that we dare not risk them by learning from the areas where we've failed, or even admitting that they exist.
    – Shog9
    Commented Apr 20, 2017 at 18:29
  • 8
    @Shog9 What I know doesn't work for gauging effectiveness of edits is to analyze characters added. I know I haven't been contributing to docs since that change (That I could rewrite a topic entirely and make it awesome and still get nothing because I maybe removed more cruft then I added). Not because I'm consumed by hunger for internet points, but because it feels like poor, rambling edits are rewarded and those who actually put in the effort to make things decent aren't.
    – Magisch
    Commented Apr 21, 2017 at 12:37
  • 8
    Great post. I do violently object to this "fix the failed mission" narrative. Google anything programming related and you'll see that Stack Overflow succeeded wildly. The problem faced is not "what do you do after you fail?" which is really quite easy, but "what do you do after you succeed" which is a soul-crushing, vantablack abyss of infinite difficulty. As for my take on docs, it is here and my mind is unchanged. Commented Apr 28, 2017 at 7:08
73

I've contributed a lot to Documentation. Ever since its launch I've wanted it to succeed but the more time I've spent with it, the more it seems to be nothing more than a rep mine.

The only people who seem to be contributing are low-level users who neither understand the material nor seem to be interested in accuracy. Most of my contributions on Docs.SO have been correcting or reverting erroneous additions.

Likewise, I never find myself using it. The search functionality is terrible and there's much better documentation spread across the internet. Combine this with the horrible additions done by low-rep users and it really seems like it's the blind leading the blind.

The only thing that has been useful is one example that I point new users to when they ask a question on SO proper and seem to lack essential debugging skills. Maybe that kind of information should have a place on SO.

I can't say for sure if Docs is a failure since I don't have the numbers but I can say that, in my experience, the examples are very poor quality. There's far more users who don't understand the topics trying to contribute then there are source material experts contributing and I don't expect that to ever change. At best, we have a few experts spending time fixing the errors of others rather than contributing original, useful content.

8
  • 8
    and a bunch of people upvoting... for who knows what reason
    – Kevin B
    Commented Apr 18, 2017 at 15:36
  • 6
    This is precisely my experience and as a result, I have absolutely no interest in using it. I don't contribute to it because it feels so overly gamed, I don't use it for my own needs since I am constantly hunting for relevant information due to how the topics are structured and the lack of excellent search capabilities, and I don't want to participate in the review queues since they're largely composed of reviewing bad grammar changes.
    – David L
    Commented Apr 18, 2017 at 15:39
  • 2
    The search functionality is terrible Seconded. I've tried a couple of times to find something and can't. At least if there some sort of structure to it thing might be easier to find. Commented Apr 18, 2017 at 15:39
  • 3
    The quality is indeed poor on most things, and the things that are high quality or usually not concise examples as the ideal would have it, but rather comprehensive tutorial style posts, like the one you linked. One of the reasons I've been holding off from contributing myself, is that there are not enough guidelines in place to dictate what is appropriate, or following the guidelines that are there (e.g. no tutorials), would mean getting rid of the most useful things. Commented Apr 18, 2017 at 15:40
  • 1
    I notice the example you actually found useful has been edited by 26 users with 2 major and 10 minor contributors. Some of those folks have very little in the way of SO reputation. This is exactly the sort of collaborative editing that Documentation was designed to encourage. Would you mind linking some counter-examples of where the system is failing?
    – Jon Ericson Staff
    Commented Apr 19, 2017 at 16:58
  • @JonEricson So many examples. These are just ones I've cleaned up. It doesn't include the topics which have blown up with minor examples.
    – Mike Cluck
    Commented Apr 19, 2017 at 17:17
  • 6
    @JonEricson The example I linked to is useful because it has been aggressively moderated by myself and others. If we hadn't been on top of it, it would have quickly fallen apart with misinformation.
    – Mike Cluck
    Commented Apr 19, 2017 at 17:20
  • 2
    "Can't find myself using it" Same for me. One feature that would really make me use it a lot more is if I could close questions as duplicate "pointers" to a Documentation section. But I've spoken to SO about this already and they seemed receptive, so hopefully that feature will be coming soon.
    – TylerH
    Commented Apr 19, 2017 at 18:32
45

The docs beta has been up for months and months, and the only time I have ever meaningfully interacted with it was to contribute a few edits to a couple things and wrote out a post on C++ meta-programming, admittedly just to scoop up some reputation while the system was still new and basic topics hadn't been documented yet. I'll take a wild guess that the only people still using it are people who are contributing to topics and rep farming.

Topics are copy-pasted verbatim from elsewhere, with barely a thought given to formatting or context. Obviously it's gotten somewhat better since then, but there's still a giant flaw in the entire idea of the system: it doesn't have any "documentation" that can't be found elsewhere on the internet. In addition, other documentation will likely be better written and more coherently put together since the tone and structure of the examples on Docs.SO is all over the map.

There are so many problems with the service that have already been enumerated elsewhere, and I have no confidence in it's future. My recommendation would be to tuck it away, do a big post mortem, and focus on improving the core SO experience. That, or maybe try some other crazy experimental project, since this one is clearly not going to work.

27

I found the idea interesting when documentation was announced. I had a particular library in mind when I read the announcement, one that I had used recently and that had a whole bunch of functions that had no documentation apart from their names and types of parameters. I just checked, that library doesn't have any documentation on SO, it's just proposed.

I looked a bit at some other projects I use. But the main issue there is, they already have at least reasonable, and sometimes excellent documentation. There isn't really much need for SO documentation for those.

It is also lacking structure, it's a list of topics sorted by a rather arbitrary popularity measure. Inside each topics are post sorted again by an arbitrary popularity measure. I'm not sure if it's meant to be only used by searching for specific items, but the lack of structure hurts if you just want to browse it.

I looked a bit closer at one documentation topic where the official documentation is pretty excellent. In the SO documentation I found one topic/example that was simply dangerous. The example showed how to backup the database using a specific software. Unfortunately, it didn't explain some important concepts (or anything at all, really), and was missing one half of the process. The backups created this way are actually unusable. And that example is there for more than half a year now.

I still think that creating something that isn't questions and answers, but covers a different angle is a reasonable idea. But I'm not convinced that documentation is the right concept.

3
  • 1
    Have you considered suggesting an edit (or simply removing) the dangerous topic?
    – Jon Ericson Staff
    Commented Apr 19, 2017 at 16:30
  • 1
    @JonEricson It's already deleted. I didn't mention it because I thought it was obvious that I'd try that. I also found another very obvious issue with something that was obviously meant as a comment in the introduction, in a very popular topic. Also removed now, but both were visible for significant timespans. My subjective impression is that there are far fewer eyeballs on this than I'd have guessed. Commented Apr 19, 2017 at 17:37
  • Yeah. Many people are doing review of new things (some carefully and others not), but not so many people are using Docs yet. It's a bit like those wikis and game guides for niche topics: quality is hit and miss. If one really good writer happens to get interested, you can find some real gems. But, you know, Sturgeon's law. On the plus side, people aren't yet using low-quality topics to screw up their actual projects.
    – Jon Ericson Staff
    Commented Apr 19, 2017 at 18:08
27

Another question already quoted this SO position:

Just this week, the executive team gave us a key metric for measuring success in the coming year: user growth.

This is a classical case of Goodhart's law. It's the 21st century. Basic company management is not some black witchcraft. We can put the blame for the failure of Documentation right on the shoulders of that executive team.

The only relevant way to judge Docs.SO is to consider the growth in good documentation. There's already enough written in other answers and comments about the details, but that addresses how wide the problem is - not how deep. It appears the problem goes all the way to management.

This management problem is not specific to Docs.SO. The same tension between "high user count" and "site quality" is a problem on the main site as well, and again management is directly involved by declaring what is and is not acceptable.

2
  • 1
    TIL Goodhart's Law. While I don't agree with it completely, it brings up a completely valid point. Thanks! Commented Apr 19, 2017 at 19:26
  • @Qix despite seeing all days, everyday? Octane was retired for that reason.
    – Braiam
    Commented Apr 19, 2017 at 23:25
19

TL;DR: We will be making some big changes in the coming months to address many of the concerns surrounding SO Docs, all towards creating an artifact that is useful to the community at large.

The details:

Let me start off by saying - Yes, we agree and acknowledge that SO Docs is not a success yet and has its flaws - both conceptually and implementation wise. There is a reason it is still in ‘beta’.

Now having said that, we at SO, have not given up on it. At its very core, the motivation and reasons for this new undertaking are still very well grounded in an unmet need for our developer community: there simply aren’t enough examples/applications/reusable code samples available for users at different skill levels. No matter how we go about solving it, that problem is very much a reality and still exists.

We truly believe that given the value Q&A offers to the community, example focused documentation can be a very powerful complement to it and provide our users with a richer experience.

But, is our current implementation of Docs, the right way? We have come to realize that that in fact is not the case - due largely to all of your input.

We take this feedback and criticism very seriously and have been extremely busy the last few months in better understanding the current perception of Docs and the various problems with the system. We have conducted multiple user research sessions with both users (with varied activity levels) and non users of Docs, all towards unearthing the barriers and motivations to use of the product. Among the issues that surfaced are the very valid concerns around -

  1. Structure and Organization - Can all topics be confined to the same, rigid sections? The sorting on the examples and the topics do not feel intuitive/logical.
  2. Hierarchy - Is tag-->topic-->example a little too simplistic?
  3. Quality concerns - What changes are good and what are not? Eg. Why was a plagiarism related rollback rejected?
  4. Navigation - It’s hard to know what SO Docs has (internal search fails) and even harder to navigate topics and the various sections. The browsability isn’t smooth.
  5. Context/Completeness - Just examples or code samples without accompanying context or information often makes the former incomplete. And without us having an official way to reference or attribute external material it makes it even harder to do so - resulting in plagiarism and/or inconsistent (but well meaning) ways the community has tried to work around it.

These problems are all very real, and it shows that we have a huge opportunity ahead of us. It also shows that we have our work cut out for us. Like I mentioned earlier, SO is as committed as ever to put our best foot forward with respect to crowdsourced Documentation.

We ask that you keep an open mind and watch out for these updates. We look forward to seeing whether these potentially charter a better path for the product or not - and as always, we will be depending on you to share your valuable feedback!

40
  • 33
    there simply aren’t enough examples/applications/reusable code samples available for users at different skill levels. No matter how we go about solving it, that problem is very much a reality and still exists. SE has stated this, but I don't really see any evidence to back up this assertion. I mean official documentation of most any product tend to not provide this, yes, but I've basically never come across any product, tool, language feature, library, etc. that I couldn't find lots of examples of its use by simply doing a google search on that topic.
    – Servy
    Commented Apr 18, 2017 at 16:28
  • 27
    Many of those examples found when searching will be SO questions related to that topic, which is a great way to find example usages of something. What evidence is there that this is a need not already met? What about the core SO engine doesn't meet this need? How does docs meet it any better than SO (both in quality of content and in discoverability in this specific situation)?
    – Servy
    Commented Apr 18, 2017 at 16:29
  • 8
    It might not be clear, but Vasudha has been working hard behind the scenes to discover the problems with Documentation. I personally appreciate having a completely outside observer dig into the product and root out problems that are obvious to users, but not so clear to those of us who have been working on it for over a year. The user interviews she mentioned have been especially eye-opening since they represent both active Stack Overflow users and people who mostly browse the site. Meanwhile, I've got renewed confidence that the need for something like Documentation exists.
    – Jon Ericson Staff
    Commented Apr 18, 2017 at 16:29
  • 36
    This post also doesn't even mention one of the core problems that Docs has had, and that's its incentive system. So many of the problems have stemmed from the fact that low quality or actively harmful contributions are highly incentivised, and useful/high quality contributions are not particularly well incentivised. Structuring the information in a more discoverable way is of course important, but so long as the most effective way to get rep is to just plagiarize other content, that's what people are going to keep doing.
    – Servy
    Commented Apr 18, 2017 at 16:34
  • 6
    @JonEricson I specifically said that official documentation tends to not have lots of good example, but that they're trivially easy to find by simple searches in existing 3rd party (or even 1st party) resources, such as SO, other Q/A sites, forums, tutorials, blogs, etc. Adding one more 3rd party source of such examples when lots already exist doesn't change the problem of official docs not having them while they're readily available elsewhere.
    – Servy
    Commented Apr 18, 2017 at 16:50
  • 6
    @JonEricson Are you still referring to very intro level concepts in mainstream programming languages, or are you referring to extremely complex and/or niche topics where there simply are no examples to be found on the whole internet? If you're referring to the former, I've never come across issues like that that don't have readily accessible examples, even if SO isn't one of the top results, if the latter, how does Docs solve the problem any better than SO proper?
    – Servy
    Commented Apr 18, 2017 at 17:10
  • 33
    @JonEricson: "I'm talking about the latter." Then step 1 needs to be changing the name. "Documentation" is not for "extremely complex and/or niche topics". One of the big problems with Docs.SO is that you keep using words against their well established meanings. If you're going to have some major restructuring of Docs.SO, so that it serves "extremely complex and/or niche topics" effectively, then it must use the right terminology from Day 1. And "Documentation" isn't it. Commented Apr 18, 2017 at 17:22
  • 14
    So what happens to all the low quality, poorly organized, unnecessary content that gets in while you guys work this out? I'd really like to see a full reset (with a data dump so that we can re-add the real gems) once making this useful is finally nailed down.
    – davidism
    Commented Apr 18, 2017 at 18:03
  • 33
    I feel like until SO as a whole understands why there is such frustration with docs this "it's not a success yet" approach is just going to make people more annoyed with the whole thing. Your list of 5 criticisms has been articulated ad nauseum for about 1 hour longer than Docs has been in public beta. It's sad to me it took over a year for this to be realized and acknowledged by Stack Overflow as a whole. It doesn't give me confidence in the future of Docs going forward if it takes a year for obvious issues to be recognized officially as legitimate.
    – enderland
    Commented Apr 18, 2017 at 18:06
  • 10
    @enderland: "Your list of 5 criticisms has been articulated ad nauseum for about 1 hour longer than Docs has been in public beta." Correction: some of those were well articulated before public beta. But your overall point is valid; it shouldn't require this much effort to convince the Powers That Be of the reality of Docs.SO. And it is very disconcerting that they had to take surveys of users before they could finally accept what they're users here were telling them. Commented Apr 18, 2017 at 19:19
  • 14
    @JonEricson So you anticipated a bunch of problems, and also recognized them early, but...choose not to do anything about them and just left the problematic features as they were. That's...not exactly inspiring optimism. Why should we now give you the benefit of the doubt and trust that you're going to actually make relevant improvements that will actually fix the underlying problems? Sure, acknowledging the problem is good, and I'm glad SE is finally doing that, but you have a ways to go to actually earn the trust of your user base back.
    – Servy
    Commented Apr 18, 2017 at 19:41
  • 19
    Why did the flagship feature that grabbed a large portion of developer attention over the past year and majorly affected the dynamics of the site only just now get a project manager?
    – davidism
    Commented Apr 18, 2017 at 20:54
  • 14
    @JonEricson: "I feel like folks misunderstand how collaborative software gets built." But if you look at successful collaborative platforms, more often than not what you find is that version 1.0 got a lot more right than wrong. MediaWiki is much more rich today than it was before, but the general idea of its initial release has been preserved. SO today has a lot of stuff in place that it didn't have originally, but the basic concept of Q&A remains. Docs.SO doesn't have that; whatever changes you're looking at making, if they're going to work at all, will have to be fundamental. Commented Apr 18, 2017 at 21:01
  • 22
    "maybe you mean we should never have tried in the first place" I think his point (and if not his, then mine) is that, if someone tells you that you're about to drive off a cliff, and you respond with, "don't worry, we'll spontaneously grow wings", it's disconcerting to see you drive off that cliff (for the better part of a year) before you manage to find someone who convinces you that people cannot spontaneously grow wings. From our perspective, the failure of Docs.SO as designed was predictable and obviously so. So we're concerned that it took so long to convince you of that. Commented Apr 18, 2017 at 21:04
  • 16
    I honestly fail to see what Documentation is supposed to provide above and beyond SO. If you want to know how to perform a certain task using a given technology, you search Stack Overflow. If there isn't a question about it yet, you ask one. Stack Overflow is example based documentation. Docs.SO is just... sad.
    – user4639281
    Commented Apr 19, 2017 at 19:19
18

To decide whether it is a failed experiment, you need metrics. And those metrics have been exposed here (quoting from The dawn of Documentation: a solstice update):

Just this week, the executive team gave us a key metric for measuring success in the coming year: user growth.

It is pretty clear that whatever features SO Docs have, or resists keeping, is a result of the criteria imposed by the decision makers who drive the company.

Those decision makers' interests is supposed to go hand in hand with the interests of the community :

Obviously there’s a business reason for that goal, but it serendipitously benefits the quality of Documentation as well. As Eric Raymond notes, “Given enough eyeballs, all bugs are shallow.” Similarly, the more people who use Documentation to understand their code, learn a new feature or even answer questions on Stack Overflow, the more reliable the content will become.

You will find there some reasons behind why it is sought to maximize the number of contributors and their incentive to contribute (generate user growth through : attracting new users to the platform ; lowering the bar for existing users to create content, which also attract new users by network effect). This is supposed to be good for the community as more users is, according to the quote, linked to better quality.

What I think we have learnt for now is that volume of users (whether contributing users or just end users) does not really correlate with the quality of the documentation. Some might even argue the opposite. About the user growth side of the equation, I can't tell (nor do I care, though I understand they have to monetize their work (and ours) one way or another).

I think we should just stop caring. We don't own the company, they have all legitimacy to pursue any project they see fit, any way that they see fit. If we don't like a particular project, we should just focus on what we like and let them care about their own. And be grateful for the things that we indeed find useful.

5
  • 10
    I agree that they're allowed to work on what they want, but I observe that most of the frustration is that they appear to be pouring so much effort into docs to the neglect of all the other long standing requests about their existing products. You know, all the stuff we like and care about.
    – davidism
    Commented Apr 18, 2017 at 13:07
  • 2
    This is a typical case where the interests of the company diverge from the interests of the community. They have to generate significant future cash flows for their investors (and themselves) while avoiding alienating a community who contributes freely (in both senses of the term) to their success. So the community certainly has some influential power but it doe not drive the company. Rather than entertaining or assuming some philanthropic motivation, we should be thinking about what exactly we are getting from SO and contributing, and adjust our time investment accordingly. No free lunch! Commented Apr 18, 2017 at 18:38
  • @user3743222 sure, but where do the free laborers come from that actually give Stack Overflow as a company business value? The community. It's a symbiotic relationship. Both us as users and SO as a company need to see the relationship as symbiotic.
    – enderland
    Commented Apr 19, 2017 at 2:01
  • 2
    Of course, and I think they very much do so. It just so happens that they have problems finding growth avenues which are in line with the community (actual/stated/fantasized) needs. I will always remember how the 25-year-old Zuckerberg flat out declared that people no longer had an expectation of privacy and consequently changed the privacy settings of its then 350MM users. This obviously had to do with the fact that they made a profit from people's information. Anyways, turns out the assumption was wrong and they updated their tune. I do believe SO will update its tune too, eventually. Commented Apr 19, 2017 at 2:12
  • 14
    It seems StackOverflow forgot why they beat ExpertsExchange - the focus on quality over quantity. The quantity of users that came to StackOverflow was a consequence of quality. Putting quantity first indeed does not work.
    – MSalters
    Commented Apr 19, 2017 at 10:10
16

Docs is a double nested XY problem.

First, it's a solution which we are all discussing to the root problem of "what should Stack Overflow as a company do next?" The underlying problem that spawned Docs is that question, not "is Docs a good idea?" -- whether Docs is a good idea doesn't actually matter because it's not the underlying question needing to be addressed.

Second, the reason Docs has so much pushback is that even assuming it is a good idea, the "case" for Docs hasn't been made and agreed upon. It's been presented as a solution to some "handwavy problem" but the root problem it supposedly solves isn't one which people really completely agree upon and so all the discussion is around the solution and not the actual problem.

If Docs had started with involved meta discussion around, "what are the main issues people face when finding documentation or examples online?" I think it would have had considerably more success. Because it would have ultimately been developed to solve a real problem people are facing and the solution itself would have been tuned to actually address a felt problem.

1
  • Upvoted for catchy-ness of first sentence. I thought it failed because there wasn't a high enough barrier to contribution. I often came across poorly written sections in my domain of expertise put there by people prolific doc contributors with minimal SO rep. I might edit, but it often took more time than if I had just written a section. And it was a lot harder than answering an SO question.
    – IRTFM
    Commented Apr 28, 2018 at 23:55
11

When this launched I posted Will Documentation duplicate effort and waste resources? but my fears became naught: instead of sucking away the already scarce documentation efforts it became a wasteland. It looks like an MVP that a summer intern typed some examples into, some of it right, some of it wrong, shallow and useless. If Shog9 is right and it's about experimenting, well, the experiment ran its course and it's pretty much dead.

You need to ask the why.

Why do we answer on StackOverflow? There's the "pay it forward" mechanism: today it's me who shares his knowledge, tomorrow it'll be you. Also, often I do it for myself -- I figure out some obscure stuff that I do not need every day so I won't memorize but I might need it later and so I need to put it somewhere and where better than SO? Even if I write both halves of the Q-A pair, commenters will force me to make it better. If I write an answer, I do not need to write the question but still I need to provide a legible answer. A month or two later, or even a few years later as I search for the info, I find my own. Awesome. Anyways, we have clear goals: an understandable question, an understandable, to the point answer and a well oiled machinery to refine it all.

In documentation -- basic information can be found elsewhere and complex information is simply a much better fit for Q&A. I can't find a reason to write docs here as it is not the official docs. I can find a reason to refine the official docs for sure -- we want better docs. But creating new, parallel comprehensive docs from ground up? Ain't nobody got time for that. Documentation lacks motivation, goals and refinery mechanism all of which has been ingeniously provided by the existing Q&A site. The lack of motivation is a big problem for primary docs, double, triple so for secondary like this.

6

I know I'm in the minority here, but I still believe the essential idea behind Docs.SO is a good one. I'm relatively new to programming, and I very often find myself reading official documentation and wishing it had more code examples. "Show, don't tell" is an important principle from storytelling that also applies to documentation; even a very good explanation of what a function does is incomplete without an example showing the code in place, working. Official documentation often lacks this. More than once I have given up on a library altogether because the documentation didn't meet my needs; now I make a habit of valuing helpful docs over fast performance or good features.

Documentation.SO's current usefulness is limited by poor searchability. I understand that the site's content isn't yet considered good enough to include in Google search, but the current workflow of

  1. visit StackOverflow.com
  2. click "Documentation"
  3. search for a tag
  4. click the tag
  5. search inside the tag

is a bit painful. Having a single place where we could search all of Docs.SO would help a lot; you could then filter the results on the other end to the tags you're interested in, much like MDN does with their "Topics" filter.

I recognize and respect the frustration of my fellow StackExchange users. I've only really programmed in HTML/CSS/JavaScript with a bit of Ruby; I've heard that other areas of the site are disasters, and I hope that stuff will be fixed. But Docs.SO has created some useful pages; the one on Debugging JavaScript taught me a lot, and this guide to HTML's <input> element is better and more continually useful than anything I've found anywhere else.

It feels like Docs.SO isn't useful to everyone, and in particular it seems to be not-useful to veterans of StackOverflow. That's a problem. I hope StackOverflow can fix it. But it is useful to me, and I'd be sad to see it discarded.

19
  • So if you're looking for some example usage of some function, and don't see such an example in the official docs, do you actually fail to find any examples using it when you do a web search on that function? If you're searching on commonly used items like, HTML's "input" tag, then that's going to be a "no". So Docs is attempting to solve a problem that Google has already solved. (And it's doing it, as you yourself noted, in a much worse and less usable manor.)
    – Servy
    Commented Apr 19, 2017 at 20:56
  • "Any examples"? No. But the examples I find usually aren't clear because they're oriented toward some other purpose. Docs.SO is a great place to find minimal, complete, verifiable examples. Commented Apr 19, 2017 at 20:59
  • Also, of course there are lots of examples for HTML's <input>--that's about as mainstream as you get. Finding examples of things that are more narrowly used is more difficult. I pointed out the <input> page because of the quality of its examples; it's comprehensive but not excessively verbose. Commented Apr 19, 2017 at 21:02
  • Ostensibly, Docs is supposed to providing real world example of actual usages of the things that they're providing examples of, not just toy snippets. This is precisely because seeing the toy snippet that doesn't do anything productive is usually what you do see out of official documentation. And of course even doing a simple search you're still likely to find reasonably minimal complete examples rather easily, so long as you're not looking into extremely niche topics. It's also worth pointing out that right now Docs isn't a good place to find such examples, it just wants to be.
    – Servy
    Commented Apr 19, 2017 at 21:02
  • And those extremely niche topics are precisely the things that you're not going to find good examples of on Docs either. Docs is attempting to solve the problem for the cases that Google already solves well (widely used, simple, mainstream topics), and isn't really working to try to solve the problem in the cases where it doesn't (extremely niche topics that see very little use).
    – Servy
    Commented Apr 19, 2017 at 21:04
  • There's a lot of documentation out there that should provide snippets, but doesn't. That's what Docs.SO is for. There's also many libraries that are poorly documented; Docs.SO provides a place where users can document that stuff if they want. You're quite correct that Docs isn't serving many of these cases well. I agree. But I think Docs can grow to serve these cases. I want to see that happen. Docs.SO is trying to fulfill a need that I feel acutely. It's far from perfect, but it's better than what we had before. Commented Apr 19, 2017 at 21:14
  • 1
    Part of what makes Docs weird is that it's intended as something that fits into gaps, like caulk into a crack. So when you look at Docs, it's like a partial map of all the places on the Internet where documentation is bad, not like a standalone edifice. Commented Apr 19, 2017 at 21:20
  • No, it's not better than what you had before. It's not even better than other options that you have now. You acting as if there is official documentation, Docs.SO, and nothing else. That's just not true. There's a whole internet out there full of examples. Everything Docs has documented and lots more already has lots of readily accessible snippets that are as good or better than what Docs has, and Docs doesn't solve that problem any better than other sources (in fact, it's worse). So the problem Docs solves isn't a problem, and the actual problems out there aren't solved by Docs.
    – Servy
    Commented Apr 19, 2017 at 21:22
  • 1
    But Docs isn't filling the gaps. It's just providing lower quality versions of what's already accessible, and not providing information not already out there. It's literally acting as the exact opposite of that.
    – Servy
    Commented Apr 19, 2017 at 21:23
  • 4
    Name a mainstream or other widely used function/operation/etc. for which you can't find working example usages in the first page of Google. Or better still, one where the only working example is Docs.
    – Servy
    Commented Apr 19, 2017 at 21:32
  • 2
    @Servy: Is your argument that SO Documentation is so terrible it will break Google? At the moment, Docs isn't working because nobody can find anything. If it ever does show up in a Google search, SO Docs has one advantage that most other search results don't have: anyone can edit the page. (Though that isn't really an advantage over MDN, I suppose.)
    – Jon Ericson Staff
    Commented Apr 19, 2017 at 22:26
  • 2
    @JonEricson I'll buy the first two, though I think it only makes 1, and that is not a bad thing. But why community-driven? why example-first? This comes already expressed as a solution, not as a problem. The question again and again is what is the problem. How much of a problem is it? What you express really seems like forcing existing SO technology into another domain independently of how much it fits the problem space. If there was no SO, and your problem was to add more/better/more realistic code examples, how would you go about it? And would you call it Code Examples or Documentation? Commented Apr 20, 2017 at 3:44
  • 2
    I upvote the answer because it addresses helpfully the question, and shows at least one person finds it useful. Though the examples given (debugging, and inputs) are all content that can be found elsewhere, whether in more simplified or detailed form - in any case they do not constitute an example of domain where documentation is lacking in examples - long or short. There might be an advantage for some users, specially beginners in having it in one place (topic). This brings the interesting question of how much value is created by structuring existing content rather than creating new content. Commented Apr 20, 2017 at 4:03
  • 1
    @user3743222: Eh. "The perfect is the enemy of the good." We've got some obvious things to do to improve the current product, which just might become good enough. We aren't starting from first principles here. The Stack Exchange community is already darn good at generating useful content and as a company we are already in the business of supporting that sort of activity.
    – Jon Ericson Staff
    Commented Apr 20, 2017 at 4:17
  • 3
    @JonEricson No, my argument is not that Docs will break Google. My argument is that Docs is trying to solve a problem that Google has already solved. Trying to provide example usages to people when they can already trivially find as many example usages as they want isn't adding value to the internet. As for them not being able to edit; a large portion of the sites that are going to show up will have a way for that person to add more information to the page.
    – Servy
    Commented Apr 20, 2017 at 13:19
-2

No. Unequivocally no.

It was a successful experiment. Successful because the experiment tested/trialled something called Stack Overflow Documentation, whose stated goal was:

"Documentation was our attempt at improving existing reference materials by focusing on examples"

The result of that successful experiment was to find that, as one can read in the other answers, SO Documentation, as implemented and run in the experiment, was unpopular and unsuccessful in achieving stated goal, failing to reach a number of hoped-for objectives in a number of metrics.

I argue that the experiment would have been unsuccessful if it had not been able to reach a conclusion about the utility of the SO Documentation project as it was implemented. In addition, based on this meta post it appears that the SO team has learned things it did not know before, complete with data to back that knowledge (see bottom of this comment). The distinction between suspected/expected/assumed/thought/rumoured/stated, and known is subtle at times, but very real. It appears that a lot of users expected this trial to be unsuccessful, but it took a successful experiment to reach that conclusion for certain.

1
  • In the vernacular sense that I asked the question, the answer is "yes". It was an experiment that failed to yield a successful product. In the scientific sense, where any experiment from which something is learned is "successful", we hope it was a success.
    – TecBrat
    Commented Oct 30, 2017 at 19:14

Not the answer you're looking for? Browse other questions tagged .