During the experiment, we observed a statistically significant 900%+ increase in clickthrough rates (CTR) in both experiment groups. Users in the experiment were engaging with related questions at an exponential rate compared to those in the control group where related questions were shown in the sidebar.
Why was this metric chosen? Were any other metrics monitored, such as whether this resulted in questions being more likely to receive answers, or the amount of time spent scrolling past this box while writing answers?
If click-through-rate is the sole metric by which changes like this are to be judged, then Stack Overflow will become useless fairly quickly. Any link will get a higher CTR if you put it in the middle of the page and make it bigger, but of course moving everything to the middle and making everything bigger will not improve the site for anybody.
This is certainly not the first time that a change in Stack Overflow's UX has been poorly received because the motivation for that change was wrong in the first place.