13

Announcing our Content Discovery initiative. In this post, we will explain what the initiative is all about, why it’s being prioritized, our initial plans, and how Meta can help us in continuing to make Stack Overflow the destination for developers and technologists.

Background

In 2022, 82% of our traffic came from organic search. For the most part, these users visit one page and bounce, never exploring further or engaging with the community. Some are finding the information they wanted, and we certainly want them to find that content more easily. However, we think surfacing more related content to their problem and making it easier to discover content better serves users so that Stack Overflow can continue to be a destination for ongoing learning and exploration.

Site satisfaction surveys have made it clear that discovering helpful content is a pain point for many visitors to Stack Overflow. Out of the 21,595 responses to the survey year to date, only 11% of participants mentioned that discoverability was one of the most valuable aspects of Stack Overflow. However, when asked “What would you most like to improve about using Stack Overflow?”, discoverability ranked third in responses, indicating that it’s important to prioritize.

To explore how we can improve in that area, we are launching an initiative focused on Content Discovery. This will start with a series of experiments to help us understand how we might help users find content more easily. We believe improving this experience will help visitors get more out of Stack Overflow every time they visit. The learnings may also surface opportunities to address other pain points within the user journey and can lead to more users engaging and contributing in the community.

Content discovery is a broad topic and we have a lot of avenues that we can take toward the goals of providing a better holistic experience and guiding folks to the information they need to find. Historically, we have also heard from Meta about the pain points and frustrations with using the embedded site search to find information and rely heavily on Google to fulfill the need. From prior research, we know that users ping pong between Stack Overflow and searching on Google to further refine their query until they are able to find an answer. How might we be able to help make it easier to do so on the platform?

Why is this being prioritized?

This initiative builds on learnings from the Outdated Answers initiative, which focused on helping users find the most current and relevant information. The primary goal for this initiative is to make it easier for users to find content that addresses their needs. There are several product development teams at Stack Overflow and we have a dedicated team focused on improving the general experience for the millions of users who visit the site every month. This team kicked off the initiative with discovery and research workshops, as well as interviews with community members which focused on their motivations for using the platform and their specific needs when they’re here.

Here are our initial plans

We're structuring this initiative by focusing on experimentation on Stack Overflow and using insights from experimentation to deepen our understanding about behavioral patterns for content consumers. Since this will have a number of small trials over time, it's worth talking about how we want to organize discussion of this initiative. This post is announcing the broader initiative and there are some related questions at the end. We will use this post as an ongoing index of the individual experiments, which will serve to collate the history of the initiative.

Content discovery experimentation

Here you will find the list of experiments that are currently running and a history of past experiments:

Currently testing:

Pending tests:

Testing concluded:

Meta literature review

Here are some posts that we’ve found on Meta while doing our initial research through the lens of finding relevant content and contributing on the platform.

Discussion

We will continue to look at current user behavior and trends, but I also want to make sure we're opening the floor for brainstorming and discussion.

  • When you land on a question page from Google, what is your thought process and behavior when you determine that you may need to explore a little more to find what you are looking for?
  • Assuming you do find what you’re looking for, what makes it more likely that you will continue reading other content that’s related to your initial need?
  • Conversely, when you are unsuccessful in finding something, what steps do you take next?

Learning more about common behaviors and patterns will help us determine where our focus should be in making it easier to find the relevant content for your needs.

17
  • 2
    Does this initiative includes researching the journey of new-to-write-code-people do after their question was closed? P.S. Just few minutes before this question I posted: Should we point new users to recent SO blog posts , podcasts episodes that mentions entities that help people to learn to code?
    – Wicket
    Oct 25, 2022 at 18:30
  • 53
    What would the... goal be? Why would someone getting here from google who found an answer that works for them... stick around? I stuck around because I was interested in answering questions, so that's one. Outside of that... what would be the purpose? so you can read questions and answers to problems you don't have?
    – Kevin B
    Oct 25, 2022 at 18:38
  • 16
    I made an account because I repeatedly stumbled over the site, but I have to agree with Kevin. A lot of people are going to bounce immediately because of a number of reasons, including the always relevant reason, because they're busy and don't have time to poke around the site. There's also people with an account who do the same thing, because SO is often a means to an end more than anything else. Trying to make users coming in from search stay by presenting more content is probably not going to be particularly efficient
    – Zoe is on strike Mod
    Oct 25, 2022 at 19:09
  • 26
    If I didn't completely and unequivocally trust the Stack company I might think this initiative is really about keeping eyeballs on the site longer for advertising dollars. Oct 25, 2022 at 19:12
  • 8
    "Historically, we have also heard from Meta about the pain points and frustrations with using the embedded site search to find information and rely heavily on Google to fulfill the need." Just wanted to pop in and say, "Thanks for hearing us and (hopefully) doing something about it". Oct 25, 2022 at 20:50
  • 11
    Does this finally mean you will make site search suck less?
    – tripleee
    Oct 26, 2022 at 7:32
  • 10
    I'm doubtful that the venn diagram of "82% of our traffic came from organic search. For the most part, these users visit one page and bounce" and "Meta participants/survey responders" has any significant crossover
    – Sam Dean
    Oct 26, 2022 at 9:22
  • 4
    If I'm looking for answers on SO, it's generally because I am busy solving some problem (i.e. working). Anything that would keep me sticking around longer than the minimum time necessary is therefore counterproductive and unwelcome -- why would you even expect that? This isn't Twitter or TikTok... | As for improving content discovery -- fix your search, and do more to prevent all the junk flooding the site, along with means to get rid of the existing mountain of garbage. That would be most useful, IMHO.
    – Dan Mašek
    Oct 26, 2022 at 17:04
  • 2
    @Rubén, not particularly but would be curious to hear more about your thoughts on how we can make it easier for folks to find relevant content while they're on the platform.
    – tanj92 Staff
    Oct 26, 2022 at 17:13
  • 4
    @KevinB, we don't have a lot of signals to tell if someone who visited one page and left were able to find their answer. We're making an assumption that a sizable amount of users aren't able to find what they're looking for which is why they bounce between Google and SO to refine their search in hope of finding an answer. If we're able to do a better job at making it easier for someone to do that by displaying more relevant content, depending on the context (i.e. if someone is looking at a specific question page vs wanting to learn more about a topic) then I'd consider that a success.
    – tanj92 Staff
    Oct 26, 2022 at 17:17
  • 3
    @tripleee search is one area that could definitely use some improvements to make it more user friendly and intuitive.
    – tanj92 Staff
    Oct 26, 2022 at 17:19
  • 7
    Speaking of content, I really wish you'd highlight the tag wiki page. It has the best content of them all -the cream of the crop, but it's well hidden from the general public.
    – TheMaster
    Oct 27, 2022 at 7:54
  • 3
  • 4
    Thanks @TheMaster! On certain tag wikis, I see there is a list of FAQs that point to SO posts, are these typically common issues that someone new to a programming language/framework could experience (e.g. if you're new to javascript, these are top 10 common issues that you could/may run into)
    – tanj92 Staff
    Oct 27, 2022 at 16:00
  • 5
    @tanj92 Yes. Links there are also often used by gold badge holders as duplicate targets.
    – TheMaster
    Oct 27, 2022 at 16:40

7 Answers 7

37

I will post a slightly cynical reply but nonetheless very true.

When you land on a question page from Google, what is your thought process and behavior when you determine that you may need to explore a little more to find what you are looking for?

My first thoughts usually are:

  • "These comments are no longer needed and only waste my time."

  • "Why isn't this not-an-answer not deleted yet?"

  • "This answer is totally wrong. I need to downvote it."

  • "This code-only answer might be what I am looking for, but the code is poorly formatted and overly verbose while at the same time there are no code comments or any explanation whatsoever."

  • "This person calls themselves a developer? How are they not ashamed to share this code publicly?"

  • "Not 'hope it helps' again!"

This is not to say that every time I open a search result I do not get what I searched for. Most of the time, I really do find the answer I wanted. And that results in an upvote. I like answers that show how to solve the issue, explain why it's the best way, and also provide a code example.

Assuming you do find what you’re looking for, what makes it more likely that you will continue reading other content that’s related to your initial need?

The overwhelming nonsense, poor formatting and web of links. That's what drags me in. In the past I ignored it, but at some point I decided I can't leave it like that. I need to start curating so that others don't have to endure the same eyesore that I had to see. I often check out the linked/related posts. I end up opening multiple tabs and I close and/or delete many of the linked questions. But I try to edit and salvage as much content as possible.

If I'm slacking off instead of doing my work, I may also look at the hot network posts to see if there's something interesting.

Conversely, when you are unsuccessful in finding something, what steps do you take next?

I change my search criteria. I search for so long that I either find what I was looking for, I end up asking a new question, or I end up finding an answer on my own and then sharing the answer on Stack Overflow with the rest.

4
  • 4
    I would say realistic, not cynical. Oct 27, 2022 at 16:36
  • 2
    82% organic search from Google because Google Search is so much better than any other. If I don't find what I am looking for I go back to the Google Search results and maybe change my search.
    – MT1
    Oct 28, 2022 at 6:13
  • 1
    How often do you come across answers that checks all three criteria where 1) shows how to solve problem 2) explain the why 3) code example provided?
    – tanj92 Staff
    Oct 28, 2022 at 18:35
  • 3
    @tanj92 Depends on what I'm searching for and how niche the topic is. But generally, the success rate of finding such answers is high. It's what makes SO great: you can find a good explanation of the solution.
    – Dharman Mod
    Oct 28, 2022 at 18:54
24

A caveat to my answers, is that I am not a new developer. I think this is going to influence my process of finding answers and how I utilize Stack Overflow to find answers compared to a newer developer.

When you land on a question page from Google, what is your thought process and behavior when you determine that you may need to explore a little more to find what you are looking for?

Generally speaking, if I'm Googling for an answer, I will put in my search and then tab-open multiple results to evaluate. Using the small snippet Google displays I can get a rough idea on how helpful the response will be. With a handful of tabs open, I start looking at the answer.

Experience, comments on the answer, and code in the answer are evaluated to determine if the answer looks directionally relevant. If it does not, I close the tab. If it does look relevant, I leave the tab open. In either case, I repeat for the handful of tabs I have open.

At the end of this process I will have 1-3 tabs still open. At that point, experience kicks in to determine which one is going to be most appropriate for my current problem. Sometimes, this is as simple as finding exact code to do a thing. More frequently, an answer will get me close - maybe it mentions a function I wasn't familiar with in a library, or points at a more appropriate alternative, etc. This is where I will need to explore more.

The important thing in this stage for me is being able to quickly eliminate the "noise" search results and identifying relevant things to continue to search for, if the answer didn't solve it completely. When I'm searching for answers, I am usually not looking for long form answers like a blog to expand on the background of a particular solution. Reading that later is helpful to learn more, but at a different time.

Assuming you do find what you’re looking for, what makes it more likely that you will continue reading other content that’s related to your initial need?

Most likely, if I've found my answer, I'm not going to be continuing to search Stack Overflow for more information to this particular problem. I probably have pulled up external documentation to learn more about a function/library/parameter/setting that I need to know more about. But, I'll read that later. This usually ends up as a tab in my browser. We all know how those multiply.

Conversely, when you are unsuccessful in finding something, what steps do you take next?

Usually the first iteration of search has given me a direction of some kind to head. It may be a very flimsy direction, but it's enough to test a couple potential ideas. Those tests and the first round of searching is used to influence another round of searching, if I still haven't solved the problem.

Additionally, don't discount utilizing coworkers, peers, and other networks to help solve a problem. Much like Stack Overflow, I can't just ask them to help. I need to provide context and the first round(s) of searching for an answer can help with that.

15

When you land on a question page from Google, what is your thought process and behavior when you determine that you may need to explore a little more to find what you are looking for?

I can't really give an answer to this from an SO perspective since I'm always here, but if I look at this from the perspective of when I land on another stack during research, for example magento.stackexchange.com, it's always for the explicit purpose of learning more about the thing I'm searching for. Rarely do I actually want a copy-paste-able answer, I want to know why the answer is the answer so that I can apply the answer to my situation myself and not need to perform that search again. I don't want to end up on the same Q&A pair a week to a month later.

The problem is... most of the time the answers don't provide that. They're usually "Here, try this:" or "You need to add this code after line 5:", and the answer was from 6 years ago so I can't even be sure that it's relevant. I can however take what is being said in those answers and use that to narrow my search or find documentation, which is where I think the real benefit lies.

Assuming you do find what you’re looking for, what makes it more likely that you will continue reading other content that’s related to your initial need?

¯\_(ツ)_/¯ When I'm searching for a solution to something... rarely am I in a position to stop what I'm doing and continue reading things that aren't relevant to the problem I'm solving. About the only thing that'd make me pause and continue reading would be seeing something wrong on the internet that needs correcting that I have the ability to correct/respond to.

Conversely, when you are unsuccessful in finding something, what steps do you take next?

Similar to Andy, I usually just open a half dozen tabs and filter through them. If one isn't presenting the info I need, I close it and go next. If none of them do, I use what I found to further restrict my search query.

12

I'm not going into answering your brainstorming questions, because I hardly do any development lately. What I do want to add though is this:

When I'm searching for a solution to a coding problem, that's usually because I'm stuck in my project. As we all know projects have deadlines, so being stuck is a situation I want to resolve as quickly as possible. That means that I'm focussed on one thing, and one thing only.

To fix that one thing I'm going to need an answer. As a not so skilled developer I'm going to be reading things regarding my problem. Generally there are three outcomes here:

  • I find out I'm facing an XY problem
  • I find out a direction towards an answer
  • I find an answer the first time

In the last case I check out the answer, and I'm going to implement it. At that point I'm gone for the day from SO. You could try to incentivize people to sign up at this point, by pointing out that they can save the answer they found in their private saves folder. This feels a bit like Firefox asking you to register an account with them to preserve your bookmarks across devices, but it could help retain users.

In the other two cases I'm going to start searching more. How useful it is that I see a search bar on top of the page now. I quickly enter a variation of the search terms I used earlier based on the insights I gained from reading the current answer. At this point I'm quickly disappointed by the poor results the search gives. This leads me to go back to Google and to retry my search there. I then might land on some advert ridden site with a tutorial on Y, or some forum style site with answers hidden behind a paywall. So as you already identified yourself, make search at least half as good as Google's would help tremendously in retaining traffic at this point.

2
  • 1
    Thanks for your sharing your experience and insights. In the other two cases where you are searching more, before you start entering keywords in the search bar, have you ever considered using either the Related or Linked modules in the sidebar?
    – tanj92 Staff
    Oct 28, 2022 at 18:32
  • 3
    @tanj92 I usually glance over them, but the title alone is hardly ever enough to make a good judgement. Titles are usually too generic, perhaps an excerpt on mouse-over would be helpful.
    – Luuklag
    Oct 29, 2022 at 6:39
12

When you land on a question page from Google, what is your thought process and behavior when you determine that you may need to explore a little more to find what you are looking for?

Typically my frustration at having to continue searching is focused on inaccurate question titles. One in particular sticks in my mind from a few years ago: trying to find out how to foo a bar, finding the perfect question based on the title "how do I foo the bar" in the right main tech stack (tagged ), only to find out it is about instead of . I think in my first year of needing to foo the bar, I landed on the winforms question at least once every time I searched how to do it to refresh my memory.

Titles are tough, and thus they are often bad. This will remain true so long as we ask users to write a title before their question. I think the title in the ask page (and in the wizard) should be at the end, not the beginning. The 2018 Ask Wizard didn't get this perfect, either, but it at least did a better job of the new one... which puts the title first again. Consider fixing that; crafting titles after crafting the question leads to better (read: more descriptive, more accurate) titles.

Assuming you do find what you’re looking for, what makes it more likely that you will continue reading other content that’s related to your initial need?

When I'm working, not much. When I'm not busy with a work task (e.g. I'm just learning or browsing), I'll click on things I want to learn about and follow the path there based on keywords (again, based on the title... everything in terms of click-throughs in discoverability comes back to that)

Conversely, when you are unsuccessful in finding something, what steps do you take next?

I go to the next tab I had open from my Google search or SO search... if I exhaust those or get through several and it becomes apparent I'm going down the wrong track, then I'll refine/replace my query in SO or Google search. Might jump over to YouTube and search there, too.

4
  • my frustration at having to continue searching is focused on inaccurate question titles definitely agree with this. I look for the list of SO results from my Google Search and try to pick the one that will be most useful. If the question title is misleading I go back to the search results and pick another. Typically the answers I need are 5 to 10 years old I don't work at the leading edge of technology!
    – MT1
    Oct 28, 2022 at 6:19
  • 1
    Do you ever read the first few lines of the question body even if you think the title is inaccurate? Or do you simply move onto the next result from your Google search that might be closer to what you're looking for?
    – tanj92 Staff
    Oct 28, 2022 at 18:26
  • 2
    @tanj92 If the title reflects what I'm looking for but it's an inaccurate title, then yes, because that's how I come to find out that the title is inaccurate. If the title does not reflect what I'm looking for, then it depends on how close it is (knowing as well as I do the difficulty of writing a good title) to what I'm trying to do. If it's somewhat close, then I'll open the link in a tab or at least read the short description shown in search results (when available).
    – TylerH
    Oct 28, 2022 at 18:35
  • Good question titles are important, and Google seems to give question titles a lot of weight. However, when people make improvement edits to question bodies they often neglect to improve the title. OTOH, it may be unwise to fix a quirky title of an old popular question, especially if it's used as a dupe target, because some people use that quirky title to find the question.
    – PM 2Ring
    Nov 1, 2022 at 15:04
6

Here are some thoughts about how I imagine a question page oriented to valuable content discovery for someone that lands on it, not signed in a SO account.

The most prominent part should be the question title, body and the "best answer". By "best answer" I'm referring to the answer with the higher score, having a positive score trending and high content health score (see below).

On second prominent place, show a score card having question relevant metrics like the number of answers, a content health score based on how is the question ageing, the activity around it both in the main site, Meta and "the third place", the chat. Also it should show metrics about answers like how many have a similar score than the "best answer", how many well received answers were posted recently, and suggested next actions like reviewing all the answers, sign-in / create an account, learn about SO, post an answer, ask a follow-up question, share & save/bookmark buttons.

On a third prominent place show the linked widget, but having links classified by relevance based on their corresponding score card. This should include links to Meta posts about the question & answers.

On fourth prominent place show the related widget ordered by "page-rank" similar to how Google sort search results.

On fifth prominent place a widget showing the tag excerpts, if the question belongs to a Collective a short description and the related articles.

On sixth prominent place a question topic related content (i.e. based on the tags) prioritized by trends, i.e. the questions having related tags being highly active

On seventh prominent place, stuff of general interest for people who write code.

On eighth prominent place a "knowledge wizard", a virtual assistant that could ask the user what they think about the question...like the question and answer clarity, relevance, complexity and helpfulness for the user's current search goal, then provide search tips or point to relevant help articles and suggest next steps, i.e. sign-in, then post on meta or answer related questions.


After reading the comments of TheMaster about giving more relevance to tag wikis, I would like to make to add a "wish":

In case that a tag-wiki be "high class" (being actively maintained, having a FAQ, having links to top-class references,...) the prominence of the corresponding widget should be moved to the third place, just below the question score card.

3
  • (Using "second most", "third most", "fourth most" might be a solution, but there may also be better ones.) Oct 27, 2022 at 16:56
  • 1
    @PeterMortensen Thank you very much for the corrections and edits on this and my others posts.
    – Wicket
    Oct 27, 2022 at 17:04
  • 2
    Thank you for all of these suggestions @Rubén. As you can tell, there's a lot of stuff we could be doing and I'm glad we are finally committing to this. I'm really interested in the tag wikis idea as it has come up in the experiment announcement post as well and it contains a lot of rich content that has been manually curated that I think could be beneficial for new learners.
    – tanj92 Staff
    Oct 28, 2022 at 18:17
4

I usually land on Stack from Google when I want to solve my current problem, as quick as possible.

I would stick around more:

  1. If the particular answer is not good enough, though, this would frustrate me.
  2. If I see a relevant and educative answer to my current problem. Especially, if I am checking the same type of error all the time, at some point, I may decide to learn it better so look for more explanatory answers. It is more like "ok I am done, teach me fishing rather than giving me a fish" approach.
  3. If I see the current answers are not good enough, then I can stay a little longer to add a better answer if I can.
1
  • 1
    Thanks for the feedback. Regarding your second point, how often does it happen where you want to learn how to fish (understand why something works in a code snippet) and does that change depending on if you are doing a personal project or for work?
    – tanj92 Staff
    Oct 28, 2022 at 18:20

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .