9

We would like to model the complete process of a user posting a new question and editing their question.

We found the data in archive.org, PostHistory.xml, which only contains the comments about the edit (for example, added 180 characters in body; edited title). We have no way to find the initial text and which parts of the information has been edited, so it is hard for us to model the change.

Enter image description here

We found that we can crawl the change from users' activity -> revision (for example, https://meta.stackoverflow.com/users/{example_ID}/{user-name}?tab=activity&sort=revisions).

Are there other ways for us to collect the text of the initial question and the text for the edited question?

We want our model to give a suggestion when the user asks a question (for example, "You should introduce a question with more background information" or "Please add the code block"). So it is easy for the user to post a good question.

3
  • 3
    Are you talking about something like this?
    – GBlodgett
    Commented Mar 28, 2019 at 4:20
  • Yes, thanks so much! Should I close or delete my question?
    – heihei
    Commented Mar 28, 2019 at 4:22
  • To find it, click on the link that says something like edited X minutes ago or edited yesterday
    – GBlodgett
    Commented Mar 28, 2019 at 4:24

2 Answers 2

7

If you have found PostHistory.xml you already have everything you need.

The initial body revision will be the rows with posthistorytypeid = 2 and any subsequent revision will be posthistorytypeid = 5 or posthistorytypeid = 8 for a rollback. The text column will contain the actual revision text.

To get the difference between two revisions in time for a specific post you can use the following query:

;with rev(pid, id, bid) as
(
select top 1000
       ph.postid pid
     , ph.id id
     , -1 bid
from posthistory ph
where ph.posthistorytypeid = 2 -- initial body
union all
select phr.postid
     , phr.id
     , fr.id
from posthistory phr
inner join rev fr on fr.pid = phr.postid
where phr.posthistorytypeid in (5,8) -- revision, rollback       
and phr.id > fr.id
)
, revall as (
select pid as postid
     , bid as baseid
     , min(id) as nextid
from rev
group by pid, bid

)

select revall.postid as [Post Link]
, len(phnext.text) - len(phbase.text) [diff]
from revall
inner join posthistory phbase on revall.baseid = phbase.id
inner join posthistory phnext on revall.nextid = phnext.id
where revall.postid = 52238064
and baseid <> -1
order by baseid

When run today your output will be similar to this:

post revisions and their diffs

The top 1000 in the first query is to reduce the result set a lot as I'm only allowed 2 minutes of processing. If you run this on your own database, leave out the top 1000 and grab a coffee.

I assume you've found Database schema documentation for the public data dump and SEDE that contains the datadictionary for the xml files found in the datadump as well as the online SEDE.

5

You're looking for the post's revision history. The URLs for those are https://stackoverflow.com/posts/<POST ID>/revisions, and the link to them is found on any post which has been edited in the form of an "edited <TIME AGO>" link next to the user's flair box.

The link

2
  • The "ago" text only seems to go up to 23 hours (seconds, minutes, and hours) and "2 days ago" (depending on GMT day start?). In between there is "yesterday" (depending on GMT day start?). And absolute time thereafter. Commented Mar 30, 2019 at 10:51
  • 1
    IIRC it's minutes up to an hour, then it's from 1-23 hours, then it's yesterday, two days ago, and finally the date. SO does, however, have a rounding problem on hours and days Commented Mar 30, 2019 at 12:32

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .