Going with an N of 5, this would mean that 5 people would have to agree that the question deserves one downvote. Most people who downvote a question agree that it was not useful or was poorly researched, and as a result will typically vote to close that question.
At this point the question is closed, with a -1. No more answers may be posted, so that it has -1 or -50 doesn't really matter anymore from that angle, because the system directly prevents anyone from posting an answer.
I know the angle for this was to improve the overall quality of the site, but I do not think that ratcheting downvotes on questions does that. Downvotes on questions are essential for both screening bad content and screening users who continuously post bad content.
It honestly pains me when I see someone like Jon Skeet (I have seen this) edit a mediocre question and move on not answering it because of other issues it contains. Avoiding wasting the time of people whose time goes out to helping many users should be, in my opinion, at the forefront of consideration for changes to the site.