Photo from Chile

Checking for JavaScript Errors with Selenium

I've recently been writing some end-to-end tests for ValidateThis, my validation framework for ColdFusion objects, using Selenium. One of the things I wanted to test were the client-side validations that the framework generates automatically. Selenium makes it very easy to test these, as I can use Selenium's assertText command to locate failure messages that have been generated by the jQuery Validation plugin, using XPath, and check whether the message is what I expected.

I found, however, that I was often getting JavaScript errors when the page first loaded (because I was working on the JS and was introducing errors), so I was looking for a quick way to add an assert to my test to check whether any JS errors occurred. After some Googling I came to the conclusion that there is nothing built into Selenium to support this, but there are a number of hacks that can be used to accomplish it. I'm going to describe one of them here. Let me state again, for the record, that this is pretty hacky. I'd love to hear from others who may have better solutions.

I simply add a script to my page that will catch any JS errors by intercepting the window.onerror event:

view plain print about
1<script type="text/javascript">
2    window.onerror=function(msg){
3        $("body").attr("JSError",msg);
4    }
5</script>

This will cause an attribute called JSError with a value corresponding to the JavaScript error message to be added to the body tag of my document if a JavaScript error occurs. Note that I'm using jQuery to do this, so this specific example won't work if jQuery fails to load. Then, in my Selenium test, I just use the command assertElementNotPresent with a target of //body[@JSError]. Now, if any JavaScript errors occur on the page my test will fail and I'll know I have to address them first. If, for some strange reason, I want to check for a particular JavaScript error, I could use the assertElementPresent command with a target of //body[@JSError='the error message'].

Note that I'm using this with a test fixture page, it's not an actual page that is part of an application. I'm not sure that this would be very useful in an automated testing environment, I'm just using it for some TDDing, where I want to write an end-to-end acceptance test first, and then write my unit tests. Getting the test to fail because of a JavaScript error is a nice way, imo, of satisfying the criteria that I must have a failing test before writing code to make it work. The usefullness of this technique in terms of pure testing is questionable, but I find it useful for my purposes, so I thought I'd put it out there for others.

TweetBacks
Comments
Interesting approach! Thanks for sharing.
# Posted By Santi | 6/8/10 12:44 AM
I don't know what a test fixture page is; that said, this is a really clever approach.
# Posted By Ben Nadel | 6/10/10 1:42 PM
By test fixture page (which may be a term I just made up), I simply mean that I create a page that is only used for running my end-to-end test. It's not an actual page that's rendered by my application. So I'm not testing the full functionality of an application page, rather I'm testing a particular function in the context of a "dummy" web page.

Does that clear it up, or only make it even more confusing?
# Posted By Bob Silverberg | 6/10/10 2:26 PM
Ah, that makes sense. Thanks for the clarification.
# Posted By Ben Nadel | 6/12/10 2:23 PM
I don't think this too hacky and is a nice way make it easy for Selenium to pick up javaScript errors. The only downside I can think of is that errors hit in a pages onLoad event will still be uncaught by selenium.

From selenium documentation: "Selenium does NOT support JavaScript alerts that are generated in a page's onload() event handler. In this case a visible dialog WILL be generated and Selenium will hang until someone manually clicks OK."

We get around this (and detect other javaScript errors) by leveraging another tool called from our selenium code: AutoIt. Detailed in this blog post: http://aptdeveloper.blogspot.com/2009/11/selenium-...
# Posted By Aaron Silverman | 7/14/10 9:33 AM
very cool & good js tip, thank you very much for sharing.
# Posted By JavaScript Media Player | 7/25/10 2:18 AM
Great approach, thank you for sharing. I'm not sure if I'm missing something, but how can you get more then one error out of the attribute?
This is the tweak we did at my work in order to chain up the errors into the body attribute and then in selenium with the get_attribute method
get all of them.

<script type="text/javascript">
window.onerror=function(msg){
var previous_errors = document.getElementsByTagName("body")[0].getAttribute("JSError");
$("body").attr("JSError",previous_errors + '\n' + msg );
}
</script>
# Posted By Goran Savovski | 12/9/10 3:06 PM
That's an interesting trick, and it looks like it would work. I was never concerned with getting all of the errors, only knowing if an error occurred at all, so I didn't need something like that.
# Posted By Bob Silverberg | 12/28/10 12:33 PM
I used this to help a QA colleague. Tt worked successfully for Firefox. In Chrome, I'm told it made Selenium fail silently and hang forever. We're looking for another way to expose errors to Selenium without breaking Chrome testing
# Posted By Eric | 2/24/11 9:41 AM
Hi,
Great approach but since we are adding an attribute to the body tag here therefore it could make the page as an invalid XHTML. Or the doctype must have this attribute defined in it. Way around for this would be to store the error in an input hidden variable and check the value of this variable. OR specify to use the above mentioned code only while running selenium tests (dunno how to do it).
# Posted By Parit | 6/29/11 6:14 AM
It works well with Selenium IDE in FF, thanks a lot ;)
# Posted By cHiaLun | 9/28/11 9:41 PM
Just wondering, does this method also work for silent JS errors? Ones that don't pop up error dialogs. But can be seen in JS error console in Firebug, etc.?
# Posted By David | 11/11/11 2:20 PM
Hi,
I didn't get where to add this code. In my selenium RC script(Java) or in the main source code of the page. Please tell me
# Posted By Naveen Kumar | 2/2/12 1:26 AM
Hi, I did try something similar. What I found is that I was not able to get much error information, the error message would be "Script Error". This may be due to security restrictions: http://stackoverflow.com/questions/5913978/cryptic... / https://bugzilla.mozilla.org/show_bug.cgi?id=36389...
# Posted By fschwiet | 4/4/12 12:47 AM
Where do I need to put this code? Inside body or head? Please let me know soon. Thanks.
# Posted By Manpreet Singh | 5/10/12 12:44 PM