94

I've been cruising the C# selenium questions for quite some time now. I frequently find myself down-voting these questions and voting to close them as needing details or clarity, or needing debugging details. Before down-voting and voting to close, I will spend a larger than average amount of time asking the OP for additional information. These questions frequently go through many edits before I finally give up, vote it down, vote to close it and move on.

Any time I've tried to answer these types of questions without an excruciating amount of detail, my answers never work. I continue a back-and-forth with the OP after answering even after the initial back-and-forth requesting more information.

I feel like questions regarding problems automating web browsers using Selenium have a pretty high standard for a minimal, reproducible example — a standard that few questions seem to hold up to. This seems to be a result of the incredible amount of code between the error and the cause, which includes Selenium, HTML, CSS, JavaScript, C# and the browser being automated. It is nigh impossible to get enough information to solve their problem without an extensive amount of guesswork. So, my question is:

What should a 'minimal, reproducible example' include to make questions about browser automation with Selenium answerable?


Canonical link to this question: [What should a 'minimal, reproducible example' include for problems with automating web browsers using Selenium?](https://meta.stackoverflow.com/q/405791)

8
  • 1
    I am torn. What you have listed is what I hope for in an mcve for anything. Enough detail to start asking sensible questions and reproducing the problem. Also, most of what you wrote applies equally to most web-scraping questions I handle in whichever language. Perhaps need a similar canonical for that. That said, there can be a big gap in knowledge between OP and answerer e.g. ability with css selectors and I find often some additional support through chat room is needed. I get that last bit is not necessarily SO ethos. I mention it cos don't expect the back and forth to completely go.
    – QHarr
    Mar 6, 2021 at 23:51
  • And my own personal pet hate..... we need the url for these things, where possible, and absolutely need enough code (html/css etc) to repro. Less a problem with selenium but for web-scraping/automation event listeners and such are also important info. My point is that I think a broader amount of detail is required though the quality bar should be the same. </siderantdone>
    – QHarr
    Mar 7, 2021 at 0:01
  • Good question. I wanted to ask a similar meta question too (but in general, for non-selenium web scraping too) -- comment on question with external link last time I encounter that I can remember
    – user202729
    Mar 7, 2021 at 6:06
  • 3
    I've seen similar issues with low quality questions on powershell, and I am told the same thing happens with Javascript. I think part of the problem is the gap between intent and code. Qs are often confused about what the real problem is and where they need help. Other Qs are just asking us to debug badly written code. Mar 7, 2021 at 13:01
  • Now that you have a great set of "what makes a good MRE" in an answer, wouldn't it be great to have somewhere more official to put it? Could we add a tag-specific MRE page?
    – Shepmaster
    Mar 8, 2021 at 17:30
  • 1
    @QHarr I think the issues with Selenium go beyond those of plain web scraping. As Greg says in his answer, "There are so many ways for automation code to fail". But there is one important similarity: a high percentage of the OPs who post low quality questions on these topics are out of their depth, trying to do a task without having sufficient knowledge of the tools they are using, and that can even include the language they're coding in.
    – PM 2Ring
    Mar 30, 2021 at 10:12
  • @PM2Ring Most of that holds true for web-scraping. Web-scraping is a form of automation particularly where browser involved. What I think is more complex for QA style automation with selenium is the set-up, particularly tests. There is an overlap particularly where *selenium is used to web-scrape.
    – QHarr
    Mar 30, 2021 at 13:13
  • 1
    @QHarr By "plain web scraping" I mean downloading a web page and extracting data from it, without using Selenium or other automation tools.
    – PM 2Ring
    Mar 30, 2021 at 22:54

1 Answer 1

96

This advice applies to every technology stack that supports Selenium.


Summary

Automating a browser with Selenium marshals a lot of machinery, both physical and virtual. There are so many ways for automation code to fail. As a result, we need a large amount of information compared to many other types of questions on StackOverflow.

For All Questions
  • Include the full error message and stack trace.
  • Which version of Selenium you are using?
  • Which web driver are you using?
  • Which version of the web driver are you using?
  • Do not post images of code!
For Questions About Web Driver Crashing

(in addition to the items above)

  • Include web driver initialization code.
  • Include the console output.
For Questions About Runtime Problems With Selenium

(in addition to the items under "For All Questions")

  • Provide a minimal, representative example of the HTML you are working with.
  • Include enough C#/Python/Java/etc... to debug your Selenium code.
  • Do include screenshots of the web page.
Template For New Questions

Copy the code below and paste into new questions about Selenium. Remove the parts you feel are unnecessary.

**Error message and stack trace:**

> Copy and paste error here (prepend each new line with a '>' character)

**Selenium Info:**

* Selenium Version: 
* Web driver type: (ChromeDriver|FirefoxDriver|EdgeDriver|SafariDriver|OperaDriver)
* Web driver version: 

**Browser Info:**

* Browser type: (Chrome|Firefox|Edge|Internet Explorer|Opera|Safari)
* Browser version: 

**Web Driver Initialization Code:**

```
Copy and paste code here
```

**Console Output:**

```
Copy and paste console/command line output here
```

**HTML Code:**

```html
Copy and paste HTML from source or element inspector
```

**Selenium Code:**

```
Copy and paste your automation code here.
```
Examples of Good Questions

I found these questions to be easy to answer and understand.

Include the full error message and stack trace.

First and foremost, always, always, always include an exception message and full stack trace if you get one.

Automating a browser using Selenium is like coordinating a full orchestra. There are so many moving parts and components. The test only works when each component is working together. A failure or misstep in any component often results in errors that are difficult to debug and trace back. The person asking the question cannot fully predict where the problem lies, so we need a lot of information.

Unfortunately for Selenium questions, the error message and stack trace is just the tip of the iceberg.

Which version of Selenium you are using?

For .NET, this would be the NuGet package version. Other languages and tech stacks have their own libraries for Selenium, and the version number should be included for other tech stacks as well.

Which web driver are you using?

Sometimes ChromeDriver has a defect. Sometimes FirefoxDriver or EdgeDriver or WhateverDriver has a defect. We need to know which browser you are automating. They all act a little differently, and this can be the key to answering your question.

Which >>>version<<< of the web driver are you using?

This is equally important as which web driver. Most browser vendors have aggressive auto-upgrade policies. If something works on Monday, but you come in on Tuesday and now it doesn't work, check your browser version. The silly browser probably auto-updated, and now you need to upgrade your web driver as well. Specific browser versions are designed to work with specific web driver versions.

Include web driver initialization code.

Don't forget to include the code you use to initialize your web driver object. Each browser vendor offers a myriad of flags you can pass to the browser when starting it. Running the browser in headless mode on a Linux server will work differently than running it with a browser window open on a Windows machine — even if you use Firefox in both environments. This initialization code can be both the source of the problem and the solution. So please include that.

Include the console output.

Most web drivers output debug statements, warnings and errors to the command line or standard output. Be sure to include those in your question, ideally formatted as a code block. This gives people answering your question more information about what goes on at runtime.


This next part is not required when the web driver crashes upon initialization.

Provide a minimal, representative example of the HTML you are working with.

There are two parts to this requirement: minimal and representative. The example should be the smallest amount of HTML possible, because web pages can contain thousands of lines of HTML. Don't copy and paste the entire source code for the page. But don't just copy and paste the one HTML tag you are trying to interact with. Include all ancestor elements up to the root <html> tag for the web page:

Worst:

No HTML at all. Providing a link to the web site is great, but I want to see the HTML code in the question. I don't want to root around some unknown web site. I don't know the site. I don't trust it. I'm not visiting it. I want to see the HTML.

Furthermore, your question and its answers should be useful to future readers. Failure to include the key elements of the HTML in your question itself risks the page changing or becoming unavailable in the future, thereby reducing (or ruining) the usefulness of the Q/A to future readers.

Bad:

<button class="btn btn-danger">Delete</button>

Reason: This is like saying "I want the brown leaf on the ground" when trying to identify leaves in a forest. Most likely there are many buttons like this on the page, so there is no way for someone answering your question to accurately know their answer is targeting the correct element.

Better:

<div>
    <fieldset>
        <legend>...</legend>
        ... other HTML tags
        <button class="btn btn-danger">Delete</button>
    </fieldset>
</div>

Reason: We see some ancestor elements (elements that exist up or above the element you are interested in. Better than the previous example. Parent, grand parent and ancestor elements give us a way to isolate the specific element on the page your question is about.

Best:

<body>
    <main id="content">
        <iframe src="..." width="..." height="...">
            <html>
                <body>
                    <form action="post" method="..." class="...">
                        <div>
                            <fieldset>
                                <legend>...</legend>
                                ... other HTML tags
                                <button class="btn btn-danger">Delete</button>
                            </fieldset>
                        </div>
                    </form>
                </body>
            </html>
       </iframe>
    <main>
</body>

Reason: We get a full view of everything up the document tree in the browser. Now people answering your question can accurately recommend locators for the element in question. And yes, sometimes the element you want is inside a <frame> or <iframe>. This is a game changer for Selenium. You will likely need to switch to that frame before locating the element. I cannot tell you how many questions are suddenly solvable once we know an <iframe> or <frame> is involved.

Some other considerations for minimal, representative HTML:

  • Always include all HTML tag attributes. Always. You'll be surprised at how unique you can make an element on the page by combining ancestor elements with attributes on your target element when crafting locators in Selenium.

  • Check for sensitive information that can identify people or financial information. Don't delete the HTML tags. Replace the sensitive information with garbage characters like X's or * characters.

    This includes email addresses!

  • Format your HTML code. Don't just copy and paste. HTML gets generated by server side templates. Those templates are formatted great. But that's not the mess I see in the browser when viewing source code. Format the HTML and indent it properly in your question, even if the HTML was generated by properly formatted templates. Which brings me to the next consideration...

  • Do not post server side template code! Selenium and the web browser do not interact with server side templates. They interact with the rendered HTML sent back from the server which happens after those server side templates are evaluated.

How To Copy HTML From Browser HTML Inspector

All major browser vendors come with some pretty robust debugging tools. I see so many images of the HTML inspector when instead it is easier to copy the actual HTML. Right-click on an HTML tag in the inspector panel, and there is usually an option to copy the HTML, inner HTML and/or outer HTML. The screenshot below is from the Firefox HTML inspector.

Screenshot of Firefox HTML inspector with context menu expanded allowing you to copy the outer HTML of an element.

Just paste the HTML into your question and click the Format As Code icon, which looks like { }.

Include enough C#/Python/Java/etc... to debug your Selenium code.

Don't just post the single line of code that throws the exception. Race conditions abound with Selenium. Line 35 might be throwing the exception, but you might need an explicit wait before Line 12 so Line 35 doesn't blow up. We will never know to recommend this unless we see Line 12.

Do you use the page object model pattern? Post the code for the page model as well. You don't need to copy and paste the entire thing, but at least give us the source code in the page model for each method called that is relevant to your problem.

public class LoginPage
{
    private readonly IWebDriver driver;
    private IWebElement Username => driver.FindElement(By.Id("username"));
    private IWebElement Password => driver.FindElement(By.Id("password"));
    private IWebElement LogInButton => driver.FindElement(By.CssSelector("button[type='submit']"));

    public LoginPage(IWebDriver driver)
    {
        this.driver = driver;
    }

    public void LogIn(string username)
    {
        Username.Clear();
        Username.SendKeys(username);
        Password.SendKeys("xxx"); // Remember to exclude hard coded passwords!!!!
        Button.Click();
    }
}

Is this part of a BDD framework like Cucumber or SpecFlow (and their ilk in any tech stack)? Add the source code for the step definitions, while you are at it. There is just so much that can go wrong.

[Binding]
public class LoginSteps
{
    private readonly LoginPage loginPage;

    public LoginSteps(IWebDriver driver)
    {
        loginPage = new LoginPage(driver);
    }

    [When(@"the user is logged in as ""(.*)""")]
    public void WhenTheUserIsLoggedInAs(string username)
    {
        loginPage.LogIn(username);
    }
}

Furthermore, you probably want to include the gherkin scenario as well:

Scenario: ...
    Given ...
    When ... # <-- it blows up here
    Then ...

Do not post images of code!

I won't get into that here, but check out this question on meta for the reasons why: Why not upload images of code/errors when asking a question?.

Images of code are an automatic down-vote and close-vote from me — unless the image of code has other information relevant to the question. But even then, include a code block with that same code.

Do include screenshots of the web page.

New users will encounter a barrier with this. You will not be able to embed images in your question. That's ok. Just provide a URL or link to the image instead. Members with higher reputation can edit your question to make it an embedded image. I frequently do this. For those other members with enough reputation to edit questions, please do this as well. A picture is worth a thousand words, after all.

And don't forget to redact sensitive information in the screenshot!


Canonical link to this answer: [What should a 'minimal, reproducible example' include for problems with automating web browsers using Selenium?](https://meta.stackoverflow.com/a/405792/3092298)

25
  • 13
    Sensible advice. Having worked with Selenium, there is a lot of things that can go wrong. For example, the browser version doesn't work with the webdriver version. Or there might even be a specific issue with one version of the webdriver and one version of the browser. A lot of times, the solution is to get the latest or second latest version of both and try to match them but if a question is posted that doesn't specify which versions are used, it's harder to say why something is wrong.
    – VLAZ
    Mar 5, 2021 at 19:52
  • 3
    It looks like a link to this answer should be found on Selenium tag wiki's When using this tag section.
    – Teemu
    Mar 6, 2021 at 12:28
  • 4
    "I don't want to root around some unknown web site." - which is probably not serving the same content as when the question was written! Mar 6, 2021 at 14:32
  • 1
    This looks like a really good, complete answer. Thanks for this. One thing I would add in the Format your HTML code section is for people to use an HTML beautifier like beautifytools.com/html-beautifier.php. I have yet to find one that is perfect but it fixes the high level issues and then I tweak it from there.
    – JeffC
    Mar 6, 2021 at 15:03
  • 1
    A second thought... maybe create an empty template based on your headings. New users can copy that template and then fill it in for their question.
    – JeffC
    Mar 6, 2021 at 15:21
  • 3
    I added an example of a quick template to the end of your answer. Feel free to adjust or remove it. I just like simple templates for some things as a quick reminder for myself to make sure I don't forget anything and I think it might help others. While this answer is very good, it's also very long. A tl;dr would be nice and I think that the template/summary might fill that role here.
    – JeffC
    Mar 6, 2021 at 15:30
  • Another thought... OS and Docker [Y|N] should probably be added to the list.
    – JeffC
    Mar 6, 2021 at 15:33
  • Excellent @OP...! Applies not only to 'Selenium' but also to other Web-Automation Tags, like [imacros]... - I've already linked to your Thread from one "current" Qt...: "Click Button and fill input with Autoit" [stackoverflow.com/q/66499233/3799241]
    – chivracq
    Mar 6, 2021 at 16:36
  • 2
    Fun document. I wish there were more like this, for other topics.
    – matt
    Mar 6, 2021 at 20:44
  • 3
    Perhaps provide an example of an existing question that (mostly) follows this guideline? Mar 7, 2021 at 2:02
  • 2
    Perhaps require the minimal, representative example of the HTML to pass HTML validation? Mar 7, 2021 at 2:05
  • 1
    The FCI Section(s) could be shortened + extended (don't you need the Browser Version + OS for 'Selenium'...?) into: "Mention your FCI (= "FULL Config Info"): Selenium + Web-Driver + Browser + OS (with ALL exact Versions)." + An Example maybe...
    – chivracq
    Mar 7, 2021 at 7:54
  • 2
    @Peter Mortensen: If the markup is invalid, attempting to change it to valid markup for the purposes of asking/answering the question would serve to obfuscate the issue. (Note that this is not the same as recommending that the maintainer of the HTML fix the source markup.)
    – BoltClock
    Mar 7, 2021 at 10:59
  • 1
    @Trilarion Nothing is quickly moved on in the [selenium] and [selenium-webdriver] tags. They're just too low volume. I see this kind of stuff all the time and it's extremely rare if it ever gets closed... let alone quickly.
    – JeffC
    Mar 7, 2021 at 15:06
  • 3
    Your opinion of the standard asker on Stack Overflow is ridiculously high if you expect them to provide even 10% of this information, never mind all of it.
    – Ian Kemp
    Mar 8, 2021 at 16:21

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .