Skip to main content
3 of 3
replaced http://meta.stackoverflow.com/ with https://meta.stackoverflow.com/

Has anyone ever tried using Stack Overflow information as part of data mining?

My question is inspired by this one as well as by this article that appeared in The Atlantic a number of years ago. The OP in the linked question was asking about the best way to demonstrate your knowledge and talent to prospective employers (specifically in the context of Stack Overflow jobs). His observation was that knowledge, talent, and your resume may not always align perfectly (I've known relatively young engineers that are almost intimidatingly talented and more senior engineers that haven't done anything new in 20 years). This can be a problem because if prospective employers look only at someone's resume they can miss out on high-potential candidates or end up hiring someone who'll be disappointing in the long run (in spite of having a lot of experience).

I was thinking: our Stack Overflow profiles have a lot of information about us as developers - code samples, writing samples, questions, answers, etc. It seems like you could infer a lot about someone (e.g. communication skills, how they're likely to interact with coworkers, their technical knowledge, etc.) based purely on their Stack Overflow profile and posts.

I'm hoping that this isn't too broad of a question, but have there been any efforts to mine this data (either alone or as data points for a larger algorithm)? How predictive is it of your actual ability as a developer? How strongly do objective measures of how well you're "doing" on Stack Overflow (e.g. reputation, badges, etc.) correlate with objective measures of job performance and success?

Does selection bias (the fact that you can delete poorly-received questions and answers, for example, and the fact that you decide which questions you attempt to answer) affect the accuracy of these measurements?

In particular, has anyone tried to use this to make hiring or promotion decisions (analogous to what the article in The Atlantic that I linked to describes about data-driven hiring or what the Oakland Athletics did when they started using a sabermetric-driven approach to identify undervalued players)?