11

I've seen Is there any read offline option on the Stack Overflow Android app?, but I was looking for a similar solution on the computer

Stack Overflow is officially supposed to contain the answers to every programming question, but what good are those answers if the internet one day goes down and all websites are lost forever, or if someone prefers to work only offline?

Shouldn't there be some kind of "download offline" feature, to download every answer with a particular tag, from Stack Overflow?

Has anyone made anything like this?

15
  • 4
    Well.... you can download every answers for every tag: archive.org/details/stackexchange Commented Jun 21, 2020 at 21:14
  • @Nick that would take a lot of CTRL+Sing Commented Jun 21, 2020 at 21:14
  • 3
    You misunderstood, you can just download the whole backup from the link above... it's quite big though, like 30+GB big Commented Jun 21, 2020 at 21:16
  • @Nick ok interesting, although I cant find where to actually download it, here archive.org/details/… it only has like 3 files? Commented Jun 21, 2020 at 21:18
  • There are also stackoverflow docsets available for kapeli.com/dash that cover specific tags - no idea how up-to-date they are Commented Jun 21, 2020 at 21:21
  • Not sure where you got that link @bluejayke but this one archive.org/download/stackexchange has 365 files for me
    – rene
    Commented Jun 21, 2020 at 21:29
  • 3
    Alternatively use SEDE: data.stackexchange.com
    – rene
    Commented Jun 21, 2020 at 21:30
  • @rene only 365 files? shouldnt thre be 19 million at least? Commented Jun 21, 2020 at 21:36
  • 1
    No. You get a posts.xml. Guess what is in there?
    – rene
    Commented Jun 21, 2020 at 21:37
  • @rene conseridering its only 365 files, no idea whatsoever Commented Jun 21, 2020 at 21:37
  • @bluejayke It's a 20GB file with every single post (there's one file for each site, but obviously SO is the one you're interested in) Commented Jun 21, 2020 at 21:48
  • 7
    "but what good are those answers if the internet one day goes down and all websites are lost forever" in a walking dead like scenario where this actually happens, I guess the answers to programming questions seem irrilevant. Commented Jun 22, 2020 at 6:08
  • 5
    In a walkin'dead like scenario, get the dump of worldbuilding.stackexchange.com instead @bluejayke.
    – TGrif
    Commented Jun 24, 2020 at 19:55
  • 1
    @HugoDelsing In a situation like that we desperately need good programmers to recover the Internet, and what will they do when they'll have questions? Commented Jul 20, 2022 at 23:01
  • 1
    There's been an official blog post Introducing the Overflow Offline project, but there's no real content besides an announcement. Maybe something solid will show up soon.
    – iBug
    Commented Oct 23, 2022 at 19:38

5 Answers 5

23

There are 3 supported options for you to get content of any SE site, including Stack Overflow, and store it offline:

  1. Use the Stack Exchange quarterly datadump as found on https://archive.org/download/stackexchange (has files for all the sites around the SE network) and import the XML files in your own datastore and/or parse/filter the XML file and keep the rows you're interested in. Each xml file has multiple <row> elements where each <row> will have the attributes for that entity. So for example in the Posts.xml of Stack Overflow you'll find <row>s for each question and each answer, over 19 million in total. The Posts.xml is for that reason a 20GB file.

  2. Use the Stack Exchange Data Explorer and write a query to match your data needs and download as CSV. SEDE is refreshed once a week. You can at most fetch 50,000 records per run (Oh really?) and the query to fetch those records need to run under 2 minutes to completion.

  3. Use the Stack Exchange API. Gives you live data, but is throttled / capped per day so if you plan on fetching lots of data, you might need a couple of days. Make sure to register your app to get a key. Apply the throttle.

For option 1 and 2 there is schema documentation found in Database schema documentation for the public data dump and SEDE

Bonus: You could setup an RSS feedreader and fetch the several RSS feeds there are: What other hidden or inobvious RSS feeds are available on Stack Exchange and its sites?. That won't backfill the posts from 2008 till now but you can start building up offline content today going forward without much effort needed.

13

For reference, Stack Overflow just announced they will provide an offline version using Kiwix for coders without an internet connection, an offline dataset to provide an essential encyclopedia.

Today, we’re happy to announce a new initiative called Overflow Offline. We’re working with a non-profit, Kiwix, to ensure that an up-to-date version of our dataset is easily available for those who need it, and will work to improve its readability and reduce its size so there is less friction for end users. Unlocked Labs is one of the organizations we’ll be working with, but there are many others, spanning from the justice system, to scientific research, to university education in areas where internet access is scarce.

https://stackoverflow.blog/2022/10/20/introducing-the-overflow-offline-project/

It can be downloaded for free (80GB) here:

https://library.kiwix.org/?lang=&q=stack+overflow

1
7

Check out StackedOff, its a very good tool: https://github.com/tools4j/stacked-off

Description:

StackedOff is a Stack Exchange site indexer and search engine. Its intended use is for people who wish to access Stack Exchange Network site(s), e.g. stackoverflow.com, but do not have a reliable internet service. StackedOff uses the 'stack dump' data files made public by the Stack Exchange Network.

5

A bit outdated version can be found in Kiwix library. Whole site is more out-dated, partial (by topic) dumps are more recent. I like this offline Kiwix, because everything is ready and searchable (and compressed), interface looks nice, and it is available for all popular OSes (Windows, Linux, MacOS, Android, iOS).

In webpage linked above search for stackoverflow.com (English), it should be about 80 GB in size. You may also search for wikipedia (English) (95 GB in size) as they have a very good set of WikiPedia too. If there are several rows of both, choose all maxi or all, both words will refer to full uncut version (because there are also "all nopic", "all mini", etc).

After downloading .zim archive from library link above you can read/view it through Kiwix program itself, downloadable here.

3
  • There also exists Kiwix-Serve, which serves the downloadable content e.g. to a local network.
    – TuringTux
    Commented Jul 20, 2022 at 15:14
  • 1
    Kiwix-Serve works like a charm. Also, the current Stack Overflow download (library.kiwix.org/?lang=eng&q=overflow) seems fairly recent (and the Kiwix client can also display Wikipedia etc.), so I think it is the best solution to read Stack Overflow offline.
    – TuringTux
    Commented Jul 21, 2022 at 10:19
  • 2
    Fun fact: Stack Overflow just announced working together with Kiwix for the "Overflow Offline" project. If I read this right, that project mainly comprises of keeping the Kiwix dataset up to date, and has maybe already been running for a while, which could be the reason the dataset was so recent the last time I checked.
    – TuringTux
    Commented Oct 23, 2022 at 19:11
1

Seekoff seems very similar to StackedOff that was already mentioned in user947737's answer: The software also downloads the dump and seems to serve it to the local network.

I haven't tested it yet, but it might be worth checking out.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .