16

What is the Version Control System that Stack Overflow uses for managing revisions of questions and answers, and is there information on the architecture employed? I had come across a post on MSE about the "tools and technologies" that Stack Exchange uses in the past and GitLab is mentioned under "External Bits". I would guess that GitLab is used for this task. But in what way? Does C# code talk with ROR endpoints? How many repositories are there? One repository for the whole of Stack Overflow? One repository per question? I could be way off, of course. But I would like some input.

12
  • 2
    GitLab may be used for the application code that actually runs the site, but I'm pretty sure question revisions are stored in the main databases with everything else.
    – jonrsharpe
    Commented Dec 30, 2016 at 16:49
  • 2
    The version control system is very likely custom made, possibly helped by a 3rd party diffing tool. I don't understand what you mean in (2) at all? Appropriate in what way?
    – Pekka
    Commented Dec 30, 2016 at 16:53
  • @jonrsharpe GitLab is mentioned twice, both under "Software Development Tools" and under "External Bits". That is why I was under the impression that GitLab (and, thus, git) is in some way used for the functionality of the site as well.
    – xnakos
    Commented Dec 30, 2016 at 16:55
  • 2
    @Pekka웃 I think 2 is a "Is this where I ask this question?" type of deal. To which, xnakos, the answer is yes, you can ask this question here. Meta is for questions about the site and the software that powers the site, so you're in the right place.
    – Kendra
    Commented Dec 30, 2016 at 16:55
  • 2
    There's some information on the diffing here: meta.stackexchange.com/questions/132771/…
    – jonrsharpe
    Commented Dec 30, 2016 at 16:57
  • 6
    Git is a source control system. It's not likely to be powering post revisions - it would be complete overkill.
    – Pekka
    Commented Dec 30, 2016 at 16:57
  • 4
    all question and answers are in the Post table, and their revisions in the PostHistory table. That is basically it
    – rene
    Commented Dec 30, 2016 at 16:58
  • @rene Thank you. Does PostHistory contain whole posts or diffs?
    – xnakos
    Commented Dec 30, 2016 at 17:04
  • @xnakos whole posts, in markdown format, while the posts table contains the rendered html version
    – rene
    Commented Dec 30, 2016 at 17:05
  • @Pekka웃 Isn't "source" a bit fluid? Markdown, XML, TXT. Aren't they all "source"? Atlas by O'Reilly uses GitLab as a backend. Atlas is a tool for online collaborative editing of books, etc. It would definitely be overkill to use Git in some cases (especially without some form of caching), as you stated. But there is a threshold and I was wondering about that threshold in general.
    – xnakos
    Commented Dec 30, 2016 at 17:26
  • 10
    By "source control" I mean it has a million features SO doesn't need. Branching, tagging, myriad other operations. SO basically needs a lot of revisions, and a diffing library to display them.
    – Pekka
    Commented Dec 30, 2016 at 19:48
  • @Pekka웃 You are totally right about that. Thanks!
    – xnakos
    Commented Dec 31, 2016 at 2:17

1 Answer 1

32

The production database schema is probably different on details but what we get on SEDE is close enough to get an idea how this works.

Each site on the network will have their own database. Each database has two main tables that handle Questions and Answers:

The posts table and the posthistory table.

The posts table hold in the rows with posttypeid = 1 the questions, and in posttypeid =2 the answers.
For a row in the posts table the events that happened in history of the post are stored in the posthistory table, basically all posthistorytypeids < 10 are related to editing of a post. A revisionguid is used to keep track of records that belong to one revision.

The posts table stores the rendered html (as you see it on the site) where the posthistory table stores the raw markdown (basically how you posted it).

For information on how the actual diffing is done see the link provided by jonrsharpe, Smarter diff algorithm for revision history.

GitLab is used internally as the version control system for the code that makes up the site. It is not used in any way for tracking revisions of a post on the sites.

See also the database schema documentation for the public data dump and SEDE and the awesome blog posts from Nick Craver.

1
  • The company has a unified database, this is behind the stackexchange.com . Only ordinary mortals can not see it. If they are doing some... more sophisticated data collection and processing, they are using probably this data. Although they clearly show that they don't do it.
    – peterh
    Commented Oct 2, 2020 at 18:51

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .