It is just a hazard of participating in a niche tag. There are just not that many people that read the posts, understand your answer and vote. I participate in several such niche tags (I've written answers to [half the questions in the [tag:zodb] tag) and the voting patterns are anaemic compared to popular tags. That's just how it is. 

If *all* you do is participate in the niche tags, there *is* a badge to encourage you to keep contributing. The [Tenacious](http://stackoverflow.com/help/badges/225/tenacious) and [Unsung Hero](http://stackoverflow.com/help/badges/226/unsung-hero) badges are aimed squarely at supporting those that answer posts with low-frequency tags. If your accepted answers don't get upvotes, at least you get a badge!