I can't speak for the UI designers and their decisions, but I think that it's because the topbar menu points are in the same bar as the search on SO.
On other SE sites with the new topbar, the search is its own seperate bar, so it's smaller on SO I suppose.
I suppose they got complaints that the searchbar is too small for some users (like high reputation users) and did this to address that concern.