Using stack overflow Q/A data to perform feature analysis in the field of Computer Science

Report:

Dataset Used

Reduction & Cleaning: In this notebook I had to pre-process all questions and answers asked by people, and determine which questions can be deemed relevant. Finally, a cleaned, new dataset which only contains relevant questions is generated.

Tag Network: In this notebook I used links beteen all tags (ex. prevalance of AngularJS questions being also Javasctipt questions) to make a web depicting topic relationships

User Question Mapping: In this notebook I mapped all users who asked questions into a high-dimensional point. The metrics considered were (1) A weighted score average per question (2) a simple counter of questions asked per category. It should be obvious that the resulting Matrix will be VERY sparse. In the end, the count metric ended up winning, but in the future, score should definitely a factor.

Score-Question Relation: In this notebook I examined a relation between the number of questions a user asked, and the overall scores they got from the questions. It turned out that a score is a very turbulent factor, and that it doesn't speak a person's expertise very well i.e. a high score in a question doesn't necessarily mean that score reflects on their knowledge of topics. Perhaps it would make more sense with answers instead of questions.