When you upvote or downvote on something, you have a few minutes to change your mind. But when you award a bounty, it's just one click and it's done. There isn't even a confirmation!
I've just misclicked on the wrong answer (because it popped up at the top of the answers list), and ended up awarding the bounty to the wrong person, and I feel terribly bad about it.
Thanks to a generous gift from rene, this specific case has been addressed, but I still think that a grace period to retract/undo assignment of a bounty would be a nice feature to add to the site.
Obviously, to prevent abuse, the grace period should be short. It could even be like that for comment upvotes, where refreshing the page clears the grace period. The idea is to cater for users who misclick. On mobile devices, where clicking is actually tapping, misclicks have become even more likely. But it's always possible for anyone to make a mistake. With the current implementation, even if you immediately realize that you've made a mistake, there is no recourse (other than starting a new bounty, and, aside from the reputation considerations, this still results in a bounty being awarded to the wrong answer).