I haven't been closely involved with the Documentation project. I've been watching it from afar with some interest, and I pitched in here on meta for a few weeks after the public beta rolled out, but outside of a few really frustrating discussions about reputation I haven't really been a part of the design or direction. So... I'm not particularly qualified to talk about it. But hey, that's never stopped me before... And there's something really critical missing from this discussion.
See, I've been watching this thread for a couple days now, and something's really bugging me about the responses. Specifically, the responses from employees of Stack Overflow The Company. They're all... kinda myopic.
I don't mean that as a put-down; the executives, project manager and community manager involved here are all doing their best to provide accurate, specific context for how we're viewing this project, what we're trying to achieve and how we're trying to approach that. And it's all really good information - if you already know where we're coming from.
But, you probably don't know that. There's, what, a year and a half worth of really dense meta posts, blog posts, answers, comments, speculation and discussion surrounding this project now, and nobody got time for that. Also... Even if you read all of it, you kinda had to read between the lines to figure out what was really going on.
So... I'm gonna try to provide some context, and then I'll try to answer your question.
Too long, won't read
If you're already bored, stop reading this answer and go read this one. Come back only if you don't understand that answer... Or don't believe it.
What follows is my personal take on where this whole "Documentation" thing comes from, and where it's going. As is my tradition, it is long, rambling, and full of digressions... You have been warned.
The chaotic money machine
Years ago, long-time SO/SE member jmort253 pointed me to this entertaining talk from Mailchimp founder Ben Chestnut. One part in particular stuck with me, since it describes one of the primary challenges facing both Stack Overflow the Q&A site and Stack Overflow the company. It's a fun video, but for the sake of those who don't have 40 minutes to burn I'm gonna quote the relevant bit from a transcript I found on Chris Barber's blog:
An entrepreneur has an idea usually. He wants to start a company. A business is like the steam machine like you don’t know how this works. You start up a business. You're like, "Oh, if I tweak this knob, I think money comes out."
If I adjust the screw or like maybe make the pulley or something tighter like more money will come out. That's the first couple of years and then after a little while, you're like, "Wait a minute. Wait a minute. Wait a minute." Two knobs. What happens then like, "Holy shit. Wow. ..." Then, you're like, "Wow, man. I kind of get this stuff. This is kind of cool." Now, I'm going to start thinking big like Richard Branson big. You start learning about key performance indicators like my KPIs are all like knobs. I'm like him but knobs on top of knobs and I've got knobs down here and then like bam.
[...]
Things go wrong when that original entrepreneur, the creative guy, says,
"You know what? I deserve a break. I'm going to delegate now. The business is running itself. I can sort of like step back a little bit. Hands off."
[...]
The thing is, your managers back at the office like, "What do I do with this thing?" He didn’t leave a manual. I don’t operate this stuff. I just protect so I'm going to hire robots and they're going to guard it. That's what I do.
[...]
Before you know it, your whole company is thinking like managers. You're not all managers but you're thinking like managers. You're defending the money machine that you made 10 years ago. No one's making new machines. No one's looking to improve it. You're just defending, defending against competition, whatever. You're just in defense mode and even worse, the creative people at the bottom, they're like, "Wow! The only way to move up in this company is to become a manager or think like a manager."
We pretty much had the whole "programmer Q&A" thing ticking along 8 years ago. There was still a lot of optimization to be done, a lot of knob-turning and such, figuring out how to scale it up and make it run smoothly... But the basic engine was running; it worked, it made money, it generated a repository of knowledge. Not very much knowledge, but... Enough to demonstrate the concept. Six years ago, that pile of knowledge was a lot bigger... And we were starting to see that defensive thinking set in, both in the folks here on meta and within the company. Today, we have millions of users and hundreds of employees who all depend on this machine running constantly, consistently, every day 24/7. Changing anything that might threaten that operation requires getting past row after row of guards, on the site, here on meta, and within the company itself. We're all in "defense mode":
The changes that do get made have to work around or compensate for deficiencies in all of the stuff that doesn't change; the end result of this is a huge, impossibly complicated system that few fully understand and where no one can predict what changing it will actually do.
At some level, anyone who's been around more than 5 minutes knows that this is problematic, that we're ossifying, burning out, expending all of our energy just to maintain position...
Pasta on the wall
More than ever, we need to be trying new things, looking for new solutions to old problems and unmet challenges to rise to. At the same time, it has never been harder to actually try anything. The crushing presence of an immense critical audience awaits; one step more and the guard dogs will be snapping at your femoral artery.
You don't have to look hard to see the signs of this: the unchanging "Ask Question" page, ancient editor, total lack of any encouragement for experienced members to maintain aging information... Anything that might cause the machine to stall becomes untouchable. We all know it's a problem even as we're all part of the problem!
So a few years back, some folks within the company decided to push in a different direction: if we can't experiment on the main Q&A system, let's build new systems, and test ideas that way. Let's throw a whole bunch of pasta against the wall & see what sticks. Will it be mentoring? Videos? Teams? Stories? International sites? Documentation? Dancing?
And so the call went out: come up with an idea, convince your co-workers to try it, convince the exec team to try it, then convince meta. Then build it & see what happens...
Trust us: we think we don't know what we're doing
If all of this sounds kinda crazy... Maybe even a little bit irresponsible... You got the right idea. We're in defense-mode for a reason after all; a lot of the ideas folks have been proposing are kinda dangerous; if we tried 'em on Q&A, they probably would break things badly, so it shouldn't surprise anyone to see that when we try 'em in Documentation they are breaking. The big difference is... We can still fix them. And we can learn something from the experience without destroying 9 years of work in the process.
Stack Overflow Q&A today looks nothing like it did in the early days. The most popular questions on the site in early '09 were those about cartoons, hidden features, jokes, and questions about Stack Overflow the site itself! It's pretty safe to say that if you took a regular SO user today, transported them back in time and dropped them into the site during its first year... They would be horrified by the lack of focus and utter chaos that reigned supreme during those days... Then they would be delighted by the lack of restrictions... Then they would be dismayed by the lack of useful information.
We got the site we have today because we tried a bunch of stuff early on, and learned from our mistakes. We need to be able to keep doing that.
Finally, the answer to your question: has Documentation failed?
Yes! Several times, in fact. I'd give you a laundry-list, but as I said at the outset there are a bunch of folks way more qualified to talk about this than me; instead, I'll point out perhaps the most blatant example:
Reputation was utterly broken at the launch of the beta. No sugar-coating it; the rep system in place at the start of the public beta was completely unworkable, and fell apart within days. What'd we learn from this? Exactly what we'd suspected since literally the private beta of Stack Overflow back in '08: that awarding reputation for editing is really, really problematic. Which is why we never ventured to touch that hornets' nest in Q&A; it would have broken everything. With Docs, we had the chance to test it, and fix it. (I'm not at all convinced that we have it right just yet, but we have made a LOT of progress)
If you're patting yourself on the back right now for knowing ahead of time that the rep system was broken... So am I; it makes it hard to type. Also, it's a bogus attitude - neither of us knew it would fail, we just made an educated guess - now we have actual experimental evidence, copious detail on exactly how it failed. If we ever want to make concrete improvements, the latter is infinitely more useful than the former. I'm sure Jon and Kevin and Vasudha have a long list of other failures - and what we've learned from them, and what we're gonna do to avoid them in the future.
In closing
It took at least a couple of years before the "best of" Stack Overflow didn't look like a joke. An awful lot of folks gave up on it before then, either because they didn't have the time to stick it out or because it was turning into something completely different from the site they thought they'd signed up for. If we stick with Documentation that long, I fully expect to see the same thing happen... But even without that rosy future, I'm excited that we're finally making big mistakes and learning from them again, turning knobs and seeing what happens. And even if you don't ever intend to use Documentation, you should be excited about that too. Everyone needs a little bit of chaos...