10

I tried to get some information about posts through the official Stack Overflow API, but I do not want to use it in an HTML context. I would like to receive the raw data in the response. Why am I getting the data with '"&<> encoded as HTML entities?

For example. The title and close reason both have &#39; instead of ': https://api.stackexchange.com/docs/questions-by-ids#order=desc&sort=activity&ids=20974098%3B24935588&filter=default&site=stackoverflow&run=true

1 Answer 1

18

See the "Safety" section in the API's Filters doc:

Filters also carry a notion of safety, which is defined as follows. Any string returned as a result of an API call with a safe filter will be inline-able into HTML without script-injection concerns. That is to say, no additional sanitizing (encoding, HTML tag stripping, etc.) will be necessary on returned strings. Applications that wish to handle sanitizing themselves should create an unsafe filter. All filters are safe by default, under the assumption that double-encoding bugs are more desirable than script injections.

If you don't like the encoding, create a custom filter and check the make unsafe checkbox at the bottom right of the filter editor pop-over.

8
  • 1
    It's such a weird and unusual choice, but at least it's trivially disabled. Commented Jul 23, 2019 at 14:13
  • 1
    Agreed it's the wrong default. It should be the actual data by default and you should opt into getting processed data. As it is you write a script to modify a post and unless you're lucky you have no idea you're destroying posts by putting back the processed data instead of the raw data.
    – user128511
    Commented Jul 24, 2019 at 3:58
  • 1
    @gman, yeah we've all done that. But no luck should be involved because you verify your app's behavior on a variety of test cases, right? ... At least our bots didn't put any XSS into stack exchange posts. ;) Commented Jul 24, 2019 at 4:02
  • "yeah we've all done that" pretty much proves my point ;)
    – user128511
    Commented Jul 24, 2019 at 4:04
  • And for the record I did test :) But, how would I know what data to test with? In other words if my tests don't contain data that gets munged I'll still have no idea that my code is wrong. I also have no idea in the first place that data gets munged at all so effectively the only way to find out about this issue is to write the bad script and then some day, if you're lucky, notice that some posts aren't coming out correctly.
    – user128511
    Commented Jul 24, 2019 at 4:11
  • @gman, if you go to the sandboxes on Meta Stack Exchange and copy or test on the gnarliest posts, you're covered for almost everything. Commented Jul 24, 2019 at 4:14
  • @gman, Anyway, the API should allow us to edit the title or tags without having to resend the body if it's unchanged. I think there is an FR for this that you should find and upvote but I don't remember where. Commented Jul 24, 2019 at 4:17
  • Thanks Brock, tests are great. But that also requires I know about the tests. Most devs don't have time to read 100% of the docs so best to do the right thing by default than only work if the dev has happened to have stumbled on and read the important page/paragraph/sentence.
    – user128511
    Commented Jul 24, 2019 at 4:25

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .