In the past, Gravatar had an issuean issue where it was serving different images for different sizes.
However, that issue has been resolved, and doesn't seem to have resurfaced.
Another reason you could be served different images is if you recently changed your email address.
Clearing your cache should fix the image.