Automated Web UI Testing Adventures with WatiN and TFS

So...I want to get some automated UI testing in place for a web application. I'm using Visual Studio Team System and Team Foundation Server.
After a bit of research, I decide to give WatiN (http://watin.sourceforge.net/) a try. WatiN is a .Net based version of WatiR, which provides an object model for interacting with an instance of Internet Explorer.

Getting a test working on my local development environment was pretty straight forward. Here's what my first test looks like:

    public class UITests
    {
        IE ie;  // The IE class is defined by WatiN for interacting with Internet Explorer

        public UITests()
        {
            ie = new IE("http://localhost/webapp");
        }
        public void TestAdminMenuIsPresent()
        {
            var result = ie.Element(Find.ByText("Admin"));
            Assert.IsNotNull(result, "Admin menu was not found");
            Assert.AreEqual("link ", result.ClassName, "Admin menu was not found");
        }
    }

So the next challenge (and its not a small one) is to get this test running on the TFS build server.
The first step towards achieving that is to get the web site deployed to the build server's local IIS. First I tried using the Visual Studio Web Deployment Project (support for this project type is available as a download). That turned out to be troublesome and not really what I wanted. All I really needed was a simple step to copy the files that make up the web app to a directory under the IIS web root (I manually configured the virtual directory in IIS - this doesn't need to be done for each build). With help from the Google I discovered the _CopyWebApplication target which will copy the application files to a destination directory. I encountered a bug with this target though in that it doesn’t copy linked files in the project. I found a blog posting about how to correct this and added a CopyLinkedContentFiles target to copy those also. The TFSBuild project for my web application now build these targets:
<Targets>Build;_CopyWebApplication;CopyLinkedContentFiles</Targets>
and it sets the following properties to cause the files to be deployed to the IIS webroot on the build server:
<Properties>OutputPath=bin\release;WebProjectOutputDir=c:\inetpub\wwwroot\webapp\;OutDir=c:\inetpub\wwwroot\webapp\bin\</Properties>


OK so then to get the test working on the build server...
Initially I was getting an error saying that the WatiN TimeoutException type couldn’t be loaded. Copying the Watin.core.dll file to the C:\Program Files\Microsoft Visual Studio 9.0\Common7\IDE directory on the build server fixed this (although I’m not sure why this was necessary/why the DLL wasn’t being deployed or found).
Then the timeout exception was being properly thrown. So, I set the WatiN timeout to 3 minutes with the following:
Settings.WaitForCompleteTimeOut = 180; // 3 minutes

Then I was hitting the VS Test default timeout, so I set the timeout on the test to 5 minutes:
[Timeout(300000), TestMethod]

Which didn’t seem to be working, so I set the per-test timeout to 5 minutes by creating a .testrunconfig file that specified this and pointing the TFS build project at this file.
Which got me back to getting the WatiN timeout:
WatiN.Core.Exceptions.TimeoutException: Timeout while Internet Explorer busy.
Based on similar reports I’ve seen it seems it is having trouble invoking IE. The reported solution was to disable “Protected Mode” in IE. I couldn’t see any way of doing this though when IE is being run under the TFSService account. But I did discover that Windows Server 2008 Server Manager has an option to turn off “IE Enhanced Security Configuration” for Administrators. So I tried turning that off.
That seemed to do the trick, leading me to the next error:
System.Threading.ThreadStateException: The CurrentThread needs to have it's ApartmentState set to ApartmentState.STA to be able to automate Internet Explorer.
OK, so I tried adding this call in the constructor of the test class:
Thread.CurrentThread.SetApartmentState(ApartmentState.STA);

But that resulted in the following error:
Error: System.InvalidOperationException: Failed to set the specified COM apartment state..
Hmmm…it turns out that you can't change the apartment state of a thread that is already running. (So what use is the SetApartmentState method?) You have to set it before the thread starts – e.g. by using the [STAThread] attribute on the entry point method of the thread. I don’t think there’s any way of doing that with unit tests though as they are invoked by the VSTestHost process. One solution to this I read of was to do the following:

            Thread thread = new Thread(STAInit);
            thread.SetApartmentState(ApartmentState.STA);
            thread.Start();
            thread.Join(); 

So, I put that into the test class constructor and marked the method that the thread starts with (STAInit) with the [STAThread] attribute. Then I tried running the test again, locally. The result this time:
System.Runtime.InteropServices.InvalidComObjectException: COM object that has been separated from its underlying RCW cannot be used.
It seems that the handle to the IE instance, which was created in my thread initializer (STAInit) was lost before the test method was called. OK, I’ll try creating the WatiN IE object in the test method itself… Yep, that worked locally at least. Now to try it on the TFS build server…
Ugh…the test is still not running in STA apartment state. OK, I’ll try having the test itself create an STA thread.
Argh, although it works fine locally, on the server it is back to timing out (exceeded the 5 minute limit I allowed for each test).
Hmmm...switching to the build server I see a debugger dialog has popped up letting me know that VSTestHost.exe has encountered an unhandled System.Threading.ThreadStateException.
Apparently WatiN will throw this if the thread's apartment state is not set to STA. Even though I am setting that on the thread I'm calling WatiN from. I wonder why this exception being thrown from a test case is causing VSTestHost to crash now?

Translate your blog

A startup company that presented at TC50 offers to publish your blog in different languages. I guess it saves foreign language speakers from having to run it through Google translate or Windows Live Translator for themselves and your blog would be indexed in other language so more likely to be found. Perhaps I should try it out...

TC50: AlfaBetic Translates Your Blog For A Worldwide Audience, Free of Charge

Localizing text in Javascript

How do you make text that is rendered via Javascript localizable? Here I'll share how I solved this on a recent project.
Here's some Javascript code embedded in a page:

<button onclick="alert('You got me!');">Click me</button>

Changing the button label to localizable text can be done by turning this button into a server side control:

<asp:button onclick="alert('You got me!');" text="<%Resources: Button_ClickMe %>" runat="server"/>

Where the button label is now defined as a (page local) string resource called Button_ClickMe. We still need to put the "You got me!" text into a string resource and pass that to the Javascript code some how.


Fortunately ASP.Net provides some handy functionality in the ScriptManager class that allows injection of code into the generated page. The ScriptManager.RegisterExpandoAttribtute is particularly useful - it allows any arbitrary attribute on any DOM object to be assigned a value. I decided to make use of this to store localizable strings as attributes on an object in the DOM.
Because I want to be able to use this functionally across multiple pages, I have defined a new base page, GlobPage (derived from System.Web.UI.Page) for all of my localizable pages to inherit from. My GlobPage class adds an empty Div control to the page to act as the placeholder for storing localized strings.

protected override void OnInit(EventArgs e)
{
LocTextContainer = new HtmlGenericControl("div");
LocTextContainer.ID = "LocalizedTextContainer";
Page.Controls.AddAt(1, LocTextContainer);
base.OnInit(e);
}

GlobPage also defines the following function to add localized string values to the LocalizedTextContainer Div

protected void RegisterJavascriptLocalStringResource(string key)
{
Page.ClientScript.RegisterExpandoAttribute("LocalizedTextContainer", key, (string)GetLocalResourceObject(key));
}

In the code behind of my page that holds the "Click Me" button I add the following to the Page_Load() method:
RegisterJavascriptLocalStringResource("Button_Response");

This puts the localized text that is defined on the Button_Response string resource into an attribute called "Button_Response" on the LocalizedTextContainer div in the generated page.
It can then be accessed from Javascript by simply using the expression LocalizedTextContainer.Button_Response as in the following:

<asp:button onclick="alert(LocalizedTextContainer.Button_Response);" text="<%Resources: Button_ClickMe %>" runat="server"/>

So don't be afraid to use Javascript in a globalized web application - it can be done!

Persisting globalized messages

This is the first in what I plan to be a series of tips and techniques related to globalization of applications. Let’s dive in…

Lots of applications not only show messages to the user, but want to persist them for some reason or other. Say you have a system that logs various messages to a database. These messages are later shown to a user looking at the log.
e.g. “Record X was changed by Y on dateZ”.

This may be written out by code such as:
Log(String.Format(“Record {0} was changed by {1} on {2}.”, recordName, userName, changeDate.ToString()));

If you are building a globalized application (i.e. one that can operate in multiple languages and cultures), you obviously don’t want to go writing out English text like that. So you do the obvious thing and refactor the localizable text into a string resource, right?
e.g.
Msg_RecordXChangedByYOnZ: “Record {0} was changed by {1} on {2}.”

This can then be simply translated to other languages by localizers. You then change your code to something like this:
Log(String.Format(GetResource("Msg_RecordXChangedByYonZ"), recordName, userName, changeDate.ToString()));

(GetResource here is an arbitrary method for fetching a localized string resource given a key.)

The problem now is that you are still writing a language specific string to the log (probably in the language of the user who caused the record change) but you don’t know the language of the user who is later going to be looking at these messages. Imagine the viewer's confusing at seeing messages shown in a variety of languages rather than just in their language of choice. You need some way of persisting this message in a language neutral format and then at display time (i.e. when you want to render the message) convert it to localized text in the appropriate language. Basically you need to persist the key of the string resource, and the parameter values. To solve this I defined some simple XML markup to store the information we need, to be able to reconstitute the message.

<Resource key="keyname" types="type1,type2,type3" values="val1,val2,val3"/>

The Values attribute contains a comma separated list of values to substituted into the localized string referred to by the Key attribute. (A comma may not be a safe separator character, but it’ll do for our purposes here.) Note that to be able to render the values correctly, we need to know what type they are (e.g. string, integer, float, date).

So then our code becomes:
Log(string.Format("<Resource Key=\"Msg_RecordXChangedByYOnZ\" Values=\"{0},{1},{2}\" Types=\"string,string,date\"/>", recordName, userName, changeDate.ToString()));

Now what we get persisted into the log will look something like this:
<Resource key="Msg_RecordXChangedByYOnZ" types="string,string,date" values="ProductXyz,Freddie,8/21/2008" />

Rendering this back to a language specific message for display to a user in their own language is now a matter of parsing the key name, values and types out of the XML, formatting each value according to its type and then putting it together with a String.Format. (I’m not showing the code for parsing and formatting the values here – I’ll leave that as an exercise for the reader.) So, to render the message:
String.Format(GetResource(key), formattedValues);

If you’ve been paying close attention you may have noticed there is still one issue to be fixed. The date we persisted to the log above is in whatever format was in use when the entry was being written (e.g. the current user’s culture format). We need to ensure that we persist it in a culture-invariant form so that it will be parsed correctly at render time.

The corrected code:
Log(string.Format("<Resource Key=\"Msg_RecordXChangedByYOnZ\" Values=\"{0},{1},{2}\" Types=\"string,string,date\"/>", recordName, userName, changeDate.ToString(CultureInfo.InvariantCulture)));

This approach proved very useful on a recent project where I found a lot of plain text messages being logged to a database. Now those messages can be viewed in the readers language of choice. A key benefit of this approach is that it doesn’t require any database schema changes to accommodate a wide variety of messages. Hopefully this technique will prove useful to someone else out there also.

Kicking things off here

I've spent a lot of my life developing software, and a lot of that has involved developing software for international markets. I spent a good chunk of my years in the industry at Microsoft, where as a foreigner (I moved from Australia to Seattle in 1998) working in the US on products for the world, I considered it one of my responsibilities to ensure the software we were putting out was in no way US-centric. In the process I've naturally picked up quite a bit of knowledge. Post-Microsoft my consulting has been primarily in the area of software globalization, assisting companies with rehabilitating their sorely US-centric market into a form that is friendly to users in other cultures and that can be localized to other languages. Seems like a good idea to share some of what I've learnt and observed in this domain.
Maybe something here will encourage other developers to broaden their thinking when designing and implementing software to consider the needs of users from different regions and cultures and perhaps avoid some of the all-too-common mistakes that tend to be made, which infuriate users by making them feel like second-class citizens.