33

It took some time for me to realize that meta communities like this one and SE.meta was also used as bug trackers, and platforms for discussion or evaluating new features.

Now, I do believe that the Q&A format can be used for lots of things, but is using it as a bug tracker really optimal?

I am comparing this to other software such as:

and others.

Now, the fundamental difference is that SE.meta has one question and several answers as its base functionality, while most traditional bug trackers are question and discussion, where each post can be of great length, containing code, image, references and more. The closest we come to this discussion platform would be comments, which are short and connected directly to either a question or an answer. This makes longer discussions much harder to keep track of.

In addition, features such as labeling, milestones and result (confirmed, rejected, in progress, etc.) is done by adding and removing tags. While this works to some degree, we have no history of changes, so we can not follow progress like we usually can in other software. This also plays a role when a bug is fixed or feature request is implemented, as there is no real concept of closing off anything, unless a developer posts an answer and this is accepted as the answer for the question.

Right now there are:

I am just curious why Stack Exchange chose to use meta communities as their bug trackers. Sure it keeps everything at the same place, but we lack certain functionality that would be great to have. Have other software platforms ever been considered?

5
  • 2
    Nitpick: you HAVE a history of changes. If people are adding/removing tag, all that is saved in revisions. Not saying it's ideal. But this information does exist.
    – Patrice
    Commented Dec 23, 2016 at 17:33
  • @Patrice That is true. I did not think about that. However, scrolling a GitHub/JIRA issue you can easily see progress. Using revisions would be more of a backdoor to find the same information. Commented Dec 23, 2016 at 17:35
  • again, this was just me being nitpicky. I am not saying that this history is as easily navigatable as Github. Just saying it does exist :P.
    – Patrice
    Commented Dec 23, 2016 at 17:36
  • 14
    It is not optimal as a bug tracker. Because it isn't, it is a bug reporting tool. Commented Dec 23, 2016 at 22:23
  • 3
    If only somebody related to SE would also produce bug tracking software, this could all be so easy… cough
    – deceze Mod
    Commented Dec 27, 2016 at 8:21

5 Answers 5

21

Optimal for whom?

It's probably quite optimal for SO. See, if they exposed a bug tracker, we would be able to actually track the bug. If a bug is rejected, we would know since it would be explicitly closed.

But this requires more effort on the part of Stack Overflow. They would have to expose a bug tracker to us, and since they don't tend to use ready-made solutions for externally-visible tech, they'd probably have to build one themselves. Our ability to directly file a bug into their system means that they can't just drop a community request on the floor and forget about it. It will always be there in their system. If they want to clear it away, they have to comment on it (even if it's just by closing it).

All of that requires work on their part. It ties their internal development to an externally visible piece of functionality.

It's much better for them to use meta. That way, users aren't filing bugs directly into their database. They can ignore issues that they don't want to address. And so forth; they can keep everything private.

So the current system is quite optimal from their perspective ;)

4
  • 2
    "See, if they exposed a bug tracker, we would be able to actually track the bug. If a bug is rejected, we would know since it would be explicitly closed." Wait, what? That's assuming that a dedicated bug tracker would actually have every bug responded to... Which is, quite frankly, unrealistic at best of times in most companies.
    – Adam Lear StaffMod
    Commented Dec 31, 2016 at 7:03
  • As we grow as a company, I agree that it does become harder to track issues with meta alone. Some teams are starting to use Trello boards to internally report/track stuff that there's consensus on addressing. I personally think that's not gonna be super sustainable in the long term, but we'll see how that goes.
    – Adam Lear StaffMod
    Commented Dec 31, 2016 at 7:06
  • 5
    We could set up a (possibly public) bug tracker. But there are trade offs, such as the overhead of someone posting on meta, then us having them (or someone else) duplicate the post in the bug tracker, all to arrive with about the same result - some bugs will get seen and fixed, some are by design, some are too minor to address given other priorities, some ideas are bad, some posts are just missed because there's a fair backlog in the system, etc. etc. I'm not gonna lie, this whole "this is just a way for SE to easily/deliberately ignore requests" angle here comes off as pretty insulting to me.
    – Adam Lear StaffMod
    Commented Dec 31, 2016 at 7:10
  • 2
    Re "they don't tend to use ready-made solutions for externally-visible tech": The (current) blog is an exception (though it came later). Commented Nov 24, 2021 at 12:29
8

I work closely with the developers and designer of the Documentation project. Meta isn't our only method of tracking bugs, but it is our primary method. I can't speak for other teams, but I believe much of what I say is applicable to them as well.

For history, Jeff Atwood began collecting bug reports on UserVoice. It prioritized issues based on a restricted voting system that didn't seem particularly useful. But it did collect a bunch of feedback for the developers to use in planning future work. In 2009, Jeff created the first meta which replaced UserVoice. Over time a system of tagging was built up around the core Q&A features. This allowed developers to track bugs using Stack Exchange question filters to see meta posts across the network.1

As a community manager, I often hear from people who have bug reports. Unless I can fix the problem myself, I always point people to meta. That has two immediate benefits:

  1. Often the bug has already been reported, so going to meta helps detect duplicates.
  2. Other users can give immediate feedback.

It also has an obvious drawback for the reporting user: bug reports are may get buried among many others on meta. However, it's important to note that the same problem would occur with we had some other bug tracking system because Stack Overflow is a relatively lean organization when measured by developers per user.

After a bug2 is posted on meta, it's trajectory varies widely. Some bugs are fixed even before they are posted. Other bugs languish forever. I know this is a frustrating situation. Jeff used to slap with a terse answer whenever something came along that wouldn't get an immediate fix. I find that equally frustrating, so I tend to not reply to meta posts when I'm not ready to give them my full attention.

In order to avoid overlooking bugs, the dev team assigns one person to "bug duty" each week. That person monitors meta for any new problems and goes back to fix old problems that weren't addressed. I don't know what criteria people use to find bugs (my guess is it varies) but the likely suspects are votes, age and activity. As far as I know, nobody pays much attention to bounties.

On the Documentation project, each week I suggest meta posts for the team to tackle. I look for questions on this meta tagged [documentation] [bug] -[status-*]. Then I scan through the newest posts that I might have missed and check the top scoring posts to find the most important bugs to fix. Then I do the same thing for s. I take a few of those posts and drop them on an internal Trello board so that we can talk about them in our weekly call.

Sometimes the result of our internal discussion is that we aren't going to fix the problem. In those cases, one of us will write up an answer and decline the request on meta. More often, the developer who fixes the bug will mark the request completed and write up an answer explaining the fix. Not only does that drop the question out of our filters, it also helps us be transparent about things that have changed. Finally, we do occasional updates give an overview of recent progress and let the community know about changes that weren't requested on meta.


Of course, the above doesn't answer the question of whether this system is optimal. It seems to me that the answer to that question depends entirely on your goals. For a large, community-run, programmer-focused network, I'd say the first goal is to make it easy for users to report and triage bugs. Generally votes are a good indication of which requests are bothering people, which is why I often sort by score. And since we are all familiar with the Q&A format, it's just about as easy to start a bug report as I can imagine.

However there are other possible goals for a bug system:

  • transparency
  • accountability
  • progress tracking
  • change history

Meta has a mixed record when it comes to these goals. Most often you'll see the input (in the form of meta questions) and the output (in the form of meta answers), but none of the in-between steps. We don't often let you know we are working on a problem until it's been fixed. As a result, the community doesn't know what we are up to most of the time.

Again, notice the problem isn't with the tool. Rather it's with the discipline required to make those things happen. I worked for years on a project that used a fully-featured issue tracker that integrated with our version control system. The tool promised to make problems of transparency and accountability go away. But our team, including managers, testers, programmers and system administrators alike, failed to use it as intended. As a result, our issue tracking system didn't live up to its promise. A tool is only as good as the use that it's put to.

That said, here's a blurb we put in a job ad for a VP of Product:

We like to work in public. Not only does that mean being open within the company about what the product team is building, we also try and bring the developer community in as soon as we’re able.

So keeping the community involved continues to be a priority for us. We hope to do a better job in the future.


Footnotes:

  1. We have a more flexible internal tool to filter meta posts now, but the basic functionality is the same.

  2. Or feature request. Most of what I say about one applies to the other. These are often less binary categories than we might hope.

2
  • 1
    So, if you mark something as status-planned or status-review, it is no longer visible in your filters? I assume that you have some other way of keeping track of such bugs or requests.
    – user4639281
    Commented Dec 31, 2016 at 4:42
  • 1
    @TinyGiant: It probably depends on the team and how they approach meta. For me, I don't use either of those statuses unless I'm about to bring up the meta post with the team or if I know the request is in our backlog. And even then, I don't see much value in adding those statuses as a rule. Either we are about to do the work (in which case we'll have a completed status to add soon enough) or we won't (in which case, I don't like to give false hope). Other teams might decide to filter on different statuses or track bugs in their own ways.
    – Jon Ericson StaffMod
    Commented Dec 31, 2016 at 4:51
7

meta.* isn't a bug tracker. It's more like "I have a bug" message to the developers. The SE team, do have a way to track bugs and feature requests:

The developers have a special view to see all bug / feature tags across the network.

1
  • 3
    That way is "combining all the bugs and feature requests across the network into one list of links", IIRC. It's not a dedicated bug tracker.
    – Undo Mod
    Commented Dec 25, 2016 at 20:33
5

Meta is more like triage for the real issue tracker. Meta sites receive a wide variety of posts: some are good, some are bad, and some are duplicates. The community sorts through all this, and when the dust settles, moderators add to worthy questions, which adds them to the internal tracker (Jira). For more information on this policy, see What posts should be escalated to staff using [status-review], and how do I escalate them?

At least, this is how it works now, across all meta sites. It did not work this way in 2016 when the question was asked.

I'd say it works well enough, better than it did before, since I see questions often getting updated tags or answers from staff, even when the same issue was reported on several different sites. See also Ticket smash for [status-review] tag: Part Deux

-3

This isn't a difficult thing to keep track of even off-the-cuff; anything tagged with 'bug' will be found.

Of course, the Stack Overflow team (formerly Stack Exchange team) have a way of amalgamating all bugs found across the network into their own internal bug tracker. It very likely has something to do with the fact that is meant to be used for bugs in the first place.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .