-5

At the moment, the only way to get to your raw reputation data as a function of time is to go to https://stackoverflow.com/reputation (as mentioned in Why did I gain/lose reputation? Can I audit my reputation history?), download the file, and write your own processing routine in your language of choice to extract the data in a suitable form for analysis.

It would be great if from your reputation page:

https://stackoverflow.com/users/your-user-id/?tab=reputation

There was an option to directly download the raw data to something like *.csv, *.txt, etc...

0

2 Answers 2

9

The data is downloadable in a structured text format directly from the reputation page. Simply download that page and save it as txt file, and you can parse it anyway you want.

Converting it to a CSV is relatively painless, considering the source data is already well structured.

I'm no bash wizard (as it will become painfully obvious), but after downloading the file and saving it into reputation.txt, for example, just execute this:

grep "^-- 2" reputation.txt | cut -b 4-14,19-23,27-34 | sed -E 's/ +/,/g' | sed -E 's/,$//' > reputation.csv

If you are on a Mac, you can dispense with downloading and just copy the whole contents of that page into the clipboard, and run instead:

pbpaste | grep "^-- 2" | cut -b 4-14,19-23,27-34 | sed -E 's/ +/,/g' | sed -E 's/,$//' > reputation.csv

You'll end up with a CSV file (reputation.csv) with the date in the first column, the rep change in the second, and the rep total in the third.

5
  • 2
    For extra geek cred, replace the "copy+paste" step with a call to wget. Nov 14, 2019 at 22:21
  • I’d need to authenticate wget’s request first. Meh. I’m on a Mac where I can just do pbpaste. This was the typical “I should be doing something else” distraction.
    – yivi
    Nov 14, 2019 at 22:24
  • 2
    Cool! I didn't know that /reputation page existed. Nov 15, 2019 at 0:32
  • Thanks, I would have never managed to find this all by myself!! Unfortunately, I ran into the problem described in superuser.com/questions/1115407/… where grep is not installed on Windows. So I installed grep for windows, but I now get the following error message when I try to run your command: 'cut' is not recognized as an internal or external command, operable program or batch file.
    – am304
    Nov 15, 2019 at 10:30
  • 1
    Yeah... Windows. Haven't touched it in a while. Maybe some other gentle soul can offer a solution for Windows. Or if you are using a relatively recent version of the Beast, you could try with WSL. Come into the light, @am304!
    – yivi
    Nov 15, 2019 at 10:32
1

For completeness, I will add my own solution to this, which uses GNU Octave as a programming environment:

% Tidy up
clear all
close all
clc

% Count nb of lines in reputation text file
fid = fopen('reputation.txt');
nb_lines = 0;
tline = fgetl(fid);
while ischar(tline)
  tline = fgetl(fid);
  nb_lines = nb_lines+1;
end
fclose(fid);

% Read data from reputation text file
file_contents = cell(nb_lines,1);
k = 1;
fid = fopen('reputation.txt');
temp = fgetl(fid);
while ischar(temp)
  file_contents{k} = temp;
  temp = fgetl(fid);
  k = k+1;
end
fclose(fid);

% Extract reputation data from file contents
data = zeros(nb_lines,3);
n = 1;
for k=1:nb_lines
  temp  = file_contents{k};
  if strcmp(temp(1:2),'--') && length(temp)>25
    data(n,1) = datenum(temp(4:13),'yyyy-mm-dd'); % Date of reputation change
    data(n,2) = str2double(temp(19:24)); % Reputation change
    data(n,3) = str2double(temp(26:end)); % Total reputation
    n = n+1;
  end
endfor
max_rep = max(data(:,3));
max_idx = find(data(:,3)==max_rep,1,'last');
rep_data = data(1:max_idx,:);

% Export the data to a CSV file
Excel_offset = 693960; % see https://uk.mathworks.com/help/exlink/convert-dates-between-microsoft-excel-and-matlab.html
csvwrite('reputation.csv',[rep_data(:,1)-Excel_offset rep_data(:,[3 2])]); 

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .