2

I'm not familiar with most of the libraries that most of questions I see around here. This makes it a little difficult to find questions in which I really can give a good answer.

So, is using robots to track questions with specific features allowed in this site? If so, which conventions must I follow?

Any information about good practices is greatly appreciated, but please notice the actual point i want to make about this is if there are conventions about this or not, and, overall, if it is ALLOWED in the first place. I like to do things the right way.

In any case I won't even try building a robot until I am sure I know every rule I have to follow, and noone should.

11
  • 2
    Don't be mean to the server. If you do feel the need to automate something that scrapes content, make sure you are gentle and use an API if any exist for the content you are after. If it is running more than once every 10 minutes then you enter a sort of gray area and may end up being temporary or permanently banned.
    – Travis J
    May 6, 2015 at 21:34
  • Of course overflowing the server would be a very stupid idea. Good to know. So, now I know making more than 1 URLquery in less than 10 minutes is a bad idea.
    – SebasSBM
    May 6, 2015 at 21:37
  • 1
    Creating an engine to put any actions (automated voting, posting automated answers, etc.) isn't a good idea. Such action might be detected as coming from a robot, and the account used by it will be banned. May 6, 2015 at 21:38
  • I don't pretend to do automated actions. I just pretend to gather a list of questions with very specific features so I can start reading them faster.
    – SebasSBM
    May 6, 2015 at 21:40
  • @SebasSBM Do you mean to simply search by tags: [c++] [singleton], or keyword added search: [c++] [singleton] tcp? May 6, 2015 at 21:44
  • No. I mean search by just the tags I want (noone more) AND (featured OR something else I want to specify)
    – SebasSBM
    May 6, 2015 at 21:47
  • By the way, isn't there a tag like 'best-practices' or something like that in Meta? Also, I would create the robot using Python instead.
    – SebasSBM
    May 6, 2015 at 21:48
  • 1
    If all fails you can use SEDE which is updated weekly. Find the db schema here
    – rene
    May 6, 2015 at 21:49
  • 1
    You could do something with the public API perhaps too.
    – Flexo Mod
    May 6, 2015 at 21:55
  • So I can make any SQL query I want to the question's DB? That is GREAT! :-) Thanks, I really appreciate to know that. What a shame it seems there isn't any URL field for posts.
    – SebasSBM
    May 6, 2015 at 22:14
  • Maybe is there a feature in Query system to list the links to questions gathered with SELECT statement? I'll ask this too in a separate question.
    – SebasSBM
    May 6, 2015 at 22:20

2 Answers 2

1

Yes. Be sure to attribute under the terms of the CCBY-SA 3.0.

You can use the Stack Exchange API

https://api.stackexchange.com/docs

There isn't anything here that would preclude you from automated queries. There is rate limiting built-in, and the API lets you know when you're hitting a limit. Be thoughtful, don't hit the server unless you need to, and only pull the data you need. Standard fare routine stuff really.

You can use SEDE

https://data.stackexchange.com/

Not automated, but you have access to the database schema and can use SQL to query it.

Data dump

https://archive.org/details/stackexchange

If you don't need up-to-date information, here's a data dump that you can download and use locally.

4

That features is already baked into the site. It's called favorite tags.

6
  • yeah, well, I'm aware that SO has awesome options to filter/-order questions. But let's say (for example) that I want to have a list of questions that have only (and JUST ONLY) the tags I am familiar with, AND that are featured questions, every 3 hours. Summarizing: very specific features. Robots may help me to achieve this.
    – SebasSBM
    May 6, 2015 at 21:32
  • 2
    There are also some pretty good search tools. You could just write a basic search and copy paste.
    – ryanyuyu
    May 6, 2015 at 21:33
  • I'd like to know somemore about searching tools. Where are them?
    – SebasSBM
    May 6, 2015 at 21:34
  • You could also bake most of what you want into one of the RSS feed features that come along with the favorite tools. In your example, you could pick the tags you want and then scrape the rest from the RSS feed without needing to dig through the site. May 6, 2015 at 21:35
  • 4
    @SebasSBM the raw link was stackoverflow.com/help/searching. It does a decent job of explaining the surprisingly powerful search box.
    – ryanyuyu
    May 6, 2015 at 21:36
  • Thank you so much! I'll check it out. Anything that may help me filtering questions with advanced and powerful filters is greatly appreciated.
    – SebasSBM
    May 6, 2015 at 21:45

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .