37

Recently, David Arenburg posted a to warn users from adding the tag and instead encourage them to use the tag. This issue was brought up about a year ago as well, with a request to retag all mistagged questions. In a community effort more than a week ago we accomplished this goal for all questions related to . Unfortunately, mistagged questions keep popping up (more in this list).

This issue is bigger than just and we would therefore like to get feedback from the meta community on how to solve this issue.

Current use of the dataframes tag:

  • The number of questions tagged is 2955 (at the time of writing this post).
  • has a high co-occurrence with the (87%) and (84%) tags.
  • The tag-wiki of does not exist and the excerpt is not very informative and doesn’t quite cover the concept of dataframes in pandas.

Current use of the data.frame tag:

  • The number of questions tagged is 7699 (at the time of writing this post).
  • has a high co-occurrence with the (97%) and a minor co-occurrence with (3%).
  • The tag-wiki of is of a high quality.

At the moment we see several possible solutions:

  1. Burninate .
  2. Retag to and extend the tag-wiki of the latter such that it also covers pandas-dataframes.
  3. Synonymize and and extend the tag-wiki of the latter such that it also covers pandas-dataframes.
  4. Create and mark both & as its synonyms. The tag wiki should be language agnostic (thus should also cover the Spark dataframe).

At the moment we have a slight preference for the fourth option because the synonyms will result in automatic redirects when the wrong tag is used. But as said, we like to hear your thoughts on these solutions and possible alternatives we didn’t cover yet.

2
  • 9
    I'd go with 4. as this should automatically retag everything (?) instead of us retaggin 3K question- which is practically impractical (you see what I did there?). Commented Mar 23, 2016 at 15:33
  • 4
    These tags will (hopefully) always be used together with the language, and will be used in one form or another, so 2-4 should be the main contenders. #4 seems both practical and democratic (in the sense that it doesn't prefer either existing tag;) Commented Mar 23, 2016 at 15:42

2 Answers 2

24

This has been with the generous help of Shog9.

So what has been done so far

  1. A new tag has been created which has the following tags as its synonyms (additional synonym proposals are welcome)

    1.1.

    1.2.

    1.3.

  2. A tag-wiki was created (see below for the first draft) to describe the most common languages associated with this tag (though any additional language specific wikis are more than welcome - hence I'm making this answer a CW)

    2.1

    2.2

    2.3

As a result, all the questions that were previously tagged with either or were automatically reatagged with to rule them all.



Regarding #4, here is my draft for the tag wiki. The first paragraph should be fine for the tag wiki excerpt, I think. The non- parts need some improvement from others and edits anywhere are welcome, so I have marked this community wiki.

Personally, I think we should minimize references to external links, since they shouldn't really be necessary for such a simple topic and make it more work to maintain. Preferably, each language section will be kept short. If we really want to elaborate a ton, well, Docs.SO will be available soon.

Below this line should just be the current draft.




A data frame is a tabular data structure. Usually, it contains data where rows are observations and columns are variables of various types. While data frame or dataframe is the term used for this concept in several languages (R, the pandas library in Python, Apache Spark), table is the term used in MATLAB and SQL.

The sections below correspond to each language that uses this term and are aimed at the level of an audience only familiar with the given language.

data.frame in R

Data frames are one of the basic tabular data structures in the R language, alongside matrices. Unlike matrices, each column can be a different data type. In terms of implementation, a data frame is a list of column vectors, each of which has the same length.

Type ?data.frame for help constructing a data frame. An example:

data.frame(
  x = letters[1:5], 
  y = 1:5, 
  z = (1:5) > 3
)
#   x y     z
# 1 a 1 FALSE
# 2 b 2 FALSE
# 3 c 3 FALSE
# 4 d 4  TRUE
# 5 e 5  TRUE

Related functions include is.data.frame, which tests whether an object is a data.frame; and as.data.frame, which coerces many other data structures to data.frame. Data frames have been extended or modified to create new data structures by several R packages, including and . For further reading, see the paragraph on Data frames in the CRAN manual Intro to R


DataFrame in Python's pandas library

The pandas library in Python is the canonical tabular data framework on the SciPy stack, and the DataFrame is its two-dimensional data object. It is basically a rectangular array like a 2D numpy ndarray, but with associated indices on each axis which can be used for alignment. As in R, from an implementation perspective, columns are somewhat prioritized over rows: the DataFrame resembles a dictionary with column names as keys and Series (pandas' one-dimensional data structure) as values.

After importing numpy and pandas under the usual aliases (import numpy as np, import pandas as pd), we can construct a DataFrame in several ways, such as passing a dictionary of column names and values:

>>> pd.DataFrame({"x": list("abcde"), "y": range(1,6), "z": np.arange(1,6) > 3})
   x  y      z
0  a  1  False
1  b  2  False
2  c  3  False
3  d  4   True
4  e  5   True

DataFrame in Apache Spark

A Spark DataFrame is a distributed collection of data organized into named columns. It is conceptually equivalent to a table in a relational database or a data frame in R/Python, but with richer optimizations under the hood. DataFrames can be constructed from a wide array of sources such as: structured data files, tables in Hive, external databases, or existing RDDs. (source)

17
  • 1
    Can tag wikis contain anchor links? That way we could make a short TOC at the top so it’s clear from the beginning that both Python and R are covered by the tag.
    – poke
    Commented Mar 23, 2016 at 20:41
  • 3
    @poke: nope, just like in any other post. Stack Overflow markdown has no support for anchors (let alone TOCs).
    – Martijn Pieters Mod
    Commented Mar 23, 2016 at 22:31
  • 1
    in R, it is misleading to say "same number of elements". I can do data.frame(x=1:10, y=I(matrix(rnorm(300), nrow=10)))
    – baptiste
    Commented Mar 24, 2016 at 7:48
  • @baptiste Okay, switched to "same length". Feel free to edit.
    – Frank
    Commented Mar 24, 2016 at 13:01
  • Note: julia uses this term, too. And deedle, though they don't have any public documentation I can find.
    – Frank
    Commented Mar 28, 2016 at 19:36
  • Personally I am not convinced that Spark DataFrame requires a dataframes / spark-dataframe tags at all. Pretty much everything until now can be covered by spark-sql. These use the same engine and are completely interchangeable. In Spark 2.0+ it becomes more about apache-spark-dataset with DataFrame being just a special case (Dataset[Row]).
    – zero323
    Commented Mar 30, 2016 at 10:05
  • 3
    @zero323 Thanks for the info. We could leave it off the wiki if you think that's best. People will continue to type patterns like dataf* into the tag field and end up adding data.frame or dataframes or whatever, though, so maybe it should be kept in.
    – Frank
    Commented Mar 30, 2016 at 12:19
  • 1
    @Frank I guess you're right :) The only thing I would like to suggest in this case is to ensure that spark-dataframe is a synonym as well
    – zero323
    Commented Mar 30, 2016 at 18:11
  • @zero323 We ended up with [data.frame] instead of [dataframe], since the mods wouldn't/couldn't help us create the latter. Still think it's best to make spark-dataframe a synonym with that?
    – Frank
    Commented Apr 8, 2016 at 15:00
  • 1
    @Frank I had to think about it and I kind of feel like it may be to R-ish in appearance.In this case I would actually make it a synonym of apache-spark-sql but I am pretty sure it won't get enough votes any time soon.
    – zero323
    Commented Apr 12, 2016 at 12:43
  • 2
    Ok, everything is merged into dataframe. Put up your wiki...
    – Shog9
    Commented Apr 13, 2016 at 15:45
  • @zero323 Ok, now we've got [dataframe] instead of [data.frame] (see Shog's comment above). Shall we merge [spark-dataframe] with it?
    – Frank
    Commented Apr 13, 2016 at 15:53
  • @Frank Sounds great :)
    – zero323
    Commented Apr 14, 2016 at 8:59
  • @zero323 you will have to remove it from here first cause it can't be a synonim of two tags Commented Apr 14, 2016 at 10:24
  • 1
    @DavidArenburg OK, I'll try to figure this out. There are a few users which can vote on this synonym and I interact with most of them on SO once in a while so I think I can simply motivate them to vote.Thanks for the answer.
    – zero323
    Commented Apr 14, 2016 at 12:17
-11

Just freakning create separated tags for everything

Contrary to removing a tag from the face of the earth, creating it is easy. Just create:

Then delete . This will solve once and for all the problem r, panda, and whoever will comes later that has the bad luck of having a functionality called data(.)?frame(s)? in the future.

Think about the children!

35
  • 5
    Oh right, I forgot about that option. This sounds like a very manual process, eh? Like to "delete" dataframes, do we have to manually remove it on each question? And if you're saying r-data.frames would be created, then we'd have to do something about the preexisting data.frame tag. Also, if you do destroy these, what is to stop someone from recreating them? I see all sorts of "DO NOT USE THIS TAG" tags around, so apparently bad tags have a way of surviving our ire and SO doesn't like to blacklist them.
    – Frank
    Commented Mar 23, 2016 at 20:58
  • @Frank you say in the OP that data.frame is mostly about r, right? The other is mostly about panda? Moderators can rename a tag easily.
    – Braiam
    Commented Mar 23, 2016 at 21:02
  • 7
    This makes both tags hard to discover though, effectively hiding them for users. So people will probably just end up attempting to recreate data.frame or dataframes. So you would have to set up synonyms, at which point you end up with the same situation as you have now.
    – poke
    Commented Mar 23, 2016 at 21:05
  • 3
    as an aside, it is [pandas] not [panda] :| Commented Mar 23, 2016 at 21:10
  • 2
    @BhargavRao I like to think they are many cute little things...
    – Braiam
    Commented Mar 23, 2016 at 21:11
  • 1
    "why the heck should the system suggest me r questions in my home page?" I think Procrastinatus is suggesting that, instead of following [dataframe], which would lead to the problem you describe, you make a tab for [pandas][dataframe]. Anyways, the problem you describe happens with many other cross-lang tags already. I follow aggregate and sometimes get questions from languages I don't know highlighted because they have that tag -- no big deal.
    – Frank
    Commented Mar 23, 2016 at 21:28
  • 3
    @Frank it may be acceptable for you... but think about this: someone gets the aggregate gold badge, they can unilaterally close questions tagged as aggregate against anything.
    – Braiam
    Commented Mar 23, 2016 at 21:30
  • 4
    @Braiam There are currently two people with a gold badge for data.frame. I think you can trust experienced users who manage to get there to refrain from closing questions regarding dataframe structures in languages they are not familiar with. And even if they make such a mistake (we all can make mistakes), it's easily rectified.
    – Roland
    Commented Mar 24, 2016 at 8:17
  • 5
    You got in an excellent point there. However do note that there are similar tags like list, set, etc for other datastructures. The disadvantage of creating language specific tags include 1. New users tagging their questions with [python-dataframes] and not [python] as the former already has python in it; thereby reducing the number of people who view the tag 2. The need of 5 users in place 1 to dupe hammer when mistagged as in point 1 (due to nonavailability of gold badgers for the new tags. This is the same case with version specific tags like [python-2.7], [python-3.4],etc(1/2) Commented Mar 24, 2016 at 9:10
  • 3
    Unfortunately that is the way the site works, Unless SO really implements must have tags like on MSO, creating smaller tags will suffer from the mentioned 2 problems. There is a real need for this, However the [feature-request]s regarding this have not received much attention from the devs. (2/2) Commented Mar 24, 2016 at 9:12
  • 1
    To illustrate the point of @BhargavRao some more examples of tags that are used for different languages: grep, join, plot, merge, regex, google-maps, text or date.
    – Jaap
    Commented Mar 24, 2016 at 11:11
  • 2
    A tag is a word or phrase that describes the topic of the question. Tags are a means of connecting experts with questions they will be able to answer by sorting questions into specific, well-defined categories. - For exactly that same reason, you can also use [r][dataframe] or [pandas][dataframe] instead of [r-dataframe] or [pandas-dataframe]
    – Jaap
    Commented Mar 24, 2016 at 16:49
  • 1
    btw: I know google-maps has an api and that grep is a command line tool, but grep is for example also a function in R. If look at the 'related tags' for each of those tags I linked to, you will see they are connected to different languages.
    – Jaap
    Commented Mar 24, 2016 at 16:50
  • 2
    "dataframe" is a word that accurately and unambiguously describes the topic of a question. Why? Because a dataframe is conceptually equivalent in R, Pandas and Spark.
    – Jaap
    Commented Mar 24, 2016 at 17:15
  • 2
    @Braiam yet there is an add tag with 2.6k questions in various languages
    – rjdown
    Commented Mar 26, 2016 at 6:29

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .