There are three ways I can see this going horribly wrong:
- Requiring more attention by the community to deal with low-quality questions
- Promoting abuse of the reviews to keep users out of my Stack Overflow
- Diluting the signal to prevent new users and the community from learning
If you can find a way to implement this without falling in to any of those traps, I am all for it. And maybe someone will find a great way to do that. In the meantime I think it's more effective for us to try to focus our time on reducing the burden for signaling that content is bad, and making that bad content less visible to the users most perturbed by it.
Attention Economy
For anything on SO to be sustainable, it needs to scale. The internet is getting bigger. The amount of things to ask questions about are increasing as new technology is invented. To just keep pace with the growth of the community, the amount of committed users needs to increase at the same rate as the amount of casual users. People are feeling the strain and making suggestions like this because they feel that balance is currently off.
There were 8,220 questions asked yesterday on Stack Overflow. 887 are closed. 1,458 have a negative score. Altogether, there were almost 1,800 questions that were poorly received in one day, and that was almost 20% of the total volume of questions asked.
By comparison, the close vote review queue doesn't even grow at that rate. And the consequences of a botched close vote review aren't that severe (the close system is designed to be a temporary thing that gives the author a chance to fix their post or ask about it on meta), whereas the proposed queue would (theoretically) stop the content from being contributed in the first place (otherwise you may as well propose dumping every new question in to the close vote review queue by default).
Those of us committed to improving the content and the site have limited time. We should be looking for ways to do more with less, and that means focusing more of our attention on the places where we have the largest positive impact, and less on the places where we have the least returns for that effort.
Would this feature accomplish that?
This is MY Stack Overflow
A lot of users of this site (which can generally be extended to people everywhere) have really strong contradictory instincts. Apparently, 14 people think an index of FAQ entries is controversial. As Jeff Atwood said, 90% of all community feedback is crap. While it's important to give people a chance to voice their opinions, it's also important to make sure that they can't cause unmitigated harm in the process.
That's why we have suspensions. And audits. And moderators. And multiple reviews from different people. And all sorts of other checks in place to make sure that even if someone goes entirely off the deep end, they won't cause lasting harm to the community.
I will 100% guarantee you that there will be people who:
- Reject every question that comes in and doesn't show 'effort' by their subjective determination
- Reject every question that even has a hint of being part of a homework assignment
- Reject every question that they assume would be found by even the most basic google query (without even checking to make sure it is that easy to find an answer)
- Reject every question that doesn't have proper code formatting rather than edit it in or see if the community is willing
And these people will prevent potentially good useful content that helps build a library of detailed answers to every question about programming.
Yes, there are ways to reduce this risk, but going back to the first point, do we really want to spend more resources to review reviews that are likely made in good faith using good judgment just to weed out these bad eggs? Because unlike a close vote (where people can see the content, edit if they feel the desire, comment to help out the poster), these questions would be invisible to the community unless they exert additional effort to look for them.
Even if the system was designed to scale, will it scale while mitigating the potential harm from people who are trying to mold Stack Overflow in their own image?
Diluting the Signal
So much of what allows SO to be successful is the ability to look at the huge amount of content that exists, to analyze it, to see trends, and to figure out ways to use that information to help make it easier to do these things in the future. The whole concept of Stack Overflow was that questions matched with answers should have clear indications of their quality and whether or not they actually solve the problem. This is a two-way street -- SO learns from their users through the data, and users learn from SO as they browse around and see the signal from the community.
Look at the current issue with comments -- we have a single signal (upvoting) which makes it difficult to actually figure out which comments are worth showing/saving, and which ones are rubbish. We can make guesstimates, but that ends up pushing out potentially good content that doesn't nicely fit the imperfect algorithm. Look at the specific-question tag. Often this is a learning process for the whole community because a specific question helps spark a discussion that better refines what we do/don't want as a community. If we give the ability to prevent the borderline cases from seeing the light of day, we lose the ability to learn from those borderline cases and grow because of it.
And the new users lose out too. Even though some of us may have had bad questions, we used the feedback we got from the community to figure out how to improve our content, and ended up becoming an active member of this site in no small part because of that feedback. We are a very complicated community with a ton of contradictory rules and a lot of confusion even from regular users. New potentially great users need feedback to improve because it's confusing here, and even a great contributor may slip up at first because of how different SO is from what they're used to.
Those new users are denied the downvotes, the comments, the friendly edits, the close reasons, and most importantly the feeling that they are part of the community. And that has a huge chance of diluting the signal that we get.