5

I am working on my thesis and I need to get Stack Overflow datasets related to the "comments" ending with ".csv".

8

3 Answers 3

13

If you don't want to download the datadump and don't want to fiddle with T-SQL in SEDE you can download about 500,000 comments in one go with this query. The query makes use of the PIVOT statement to spread row values across multiple columns.

declare @limitrows int = ##limit?100000##

declare @cols int = 10 -- if you change this 
                       -- change the column labels as well
                       -- in two places !

select [row]
      -- 10 cols
      , [0]
      , [1]
      , [2]
      , [3]
      , [4]
      , [5]
      , [6]
      , [7]
      , [8]
      , [9]
from 
    (select (seq-1) % @cols [col]
    , row / @cols [row]
    , text
    from
    ( select top (@limitrows)
       row_number() over(order by id) seq
     , row_number() over(order by id) row
     , text
     from comments ) as c
     ) as ca
pivot (
   min(text)
     for ca.col in ( 
      -- 10 cols
        [0]
      , [1]
      , [2]
      , [3]
      , [4]
      , [5]
      , [6]
      , [7]
      , [8]
      , [9])
) p

You first run the query and after you got a result, click the Download CSV button at the top right of the result grid. I indicated that button with a red free hand circle in the below screenshot:

Insane result grid, showing columns 1, 2, 3 and 4. The button download csv at the top right of the grid is encircled

I leave it as an exercise for the reader to parameterize the inner query that drives which comments get selected. The answer from Oleg Valter can be helpful with that.

Keep in mind SEDE is updated once a week on Sunday.
Use the excellent SEDE Tutorial written by the admirable Monica Cellio.
Say "Hi" in SEDE chat.

1
9

As an alternative to data dumps, you can use the Stack Exchange Data Explorer (aka SEDE). Although due to sheer number of comments on Stack Overflow, such a query is going to be slow due to the leading wildcard, but can fit into the execution time limit if you forgo the ordering:

select
  *
from
  comments
where
  text like '%' + ##Ending:string##
8

Stack Exchange in general runs data dumps that are available for public consumption (list of all dumps can be found here). In particular, SO (due to the site's size) breaks data dumps into separate dumps. The March 17, 2022 dump of comments is here. You'll have to parse it for what you need, but it's the easiest way to compile the data you seek.

Good luck!

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .