Skip to main content

2 posts tagged with "coded ui"

View All Tags

Coded UI and the Curse of the Docking Station

I’ve a love / hate relationship with Coded UI. Well hate / hate might be more accurate. Hate perhaps married with a very grudging respect still underpinned by a wary bitterness. Yes, that’s about the size of it. Why? Well, when Coded UI works, it’s fab. But it’s flaky as anything. Anybody who’s used the technology is presently nodding sagely and holding back the tears. It’s all a bit... tough.

I’ve recently discovered another quirk to add to the list. Docking stations. I was back working on a project which had a Coded UI test suite. I’d heard tell that there were problems with the tests and was just taking a look at them. The first hurdle I fell at was getting the tests to run locally. The tests had first been developed on a standard desktop build and, as much as this can ever be said of Coded UI tests, they worked. However, the future had happened. The company in question was no longer using the old school desktop towers. Nope, they’d reached for the sky and equipped the whole office with Surface Pro 3’s, hot desks, docking stations and big, big monitors. It looked terribly flash.

Coded UI was not happy.

The Mouse.Click behaviour wasn’t working. Most tests need the ability for users to click on buttons, dropdowns etc. That’s part of a normal UI. And so it was with these tests. This is where they fell over. The reason they fell over at this point didn’t become clear for a while. It wasn’t until we tried tweaking our implementation of the tests that we realised what was happening. The tests normally found buttons / dropdowns etc on the screen and then attempted to perform a Mouse.Click upon them. We changed the implementation to be subtly different. Instead of just clicking on the element we amended the test to move the mouse to the button and then perform the click.

Aha!

Rather than steadily moving towards an element and clicking, the pointer was swerving like a drunk man crossing the road at 3am. It completely missed the element it was aiming for and clicked upon a seemingly random area of the screen. This is Coded UI doing “pin the tail on the donkey”.

After more time than I'd like to admit I happened upon the solution. I tended to dock my Surface and then tune my monitor resolution to the one most optimal for coding. (ie really high res.) This is what messes with Coded UI's head; the resolution change. If I wanted to be able to run tests successfully all I had to do was switch back to the resolution I initially booted with. Alternately I could restart my computer so it launched with the resolution I was presently using.

Once you do follow this guidance Coded UI has a moment of clarity, gets sober and starts Mouse.Click-ing like a pro.

Pretending to be someone you're not and the dark pit of despair

(Coded UI, IE 11 and the "runas" problem)#

"I'm not angry, I'm just disappointed."

That's kind of how I feel about Coded UI tests. It may well be that you've never heard of them - in my experience very few people seem to be aware of them. What are they? Well, I've never used Selenium but as best I understand Coded UI is Microsoft's own version of that. Namely it's a way to automate testing, in my case browser-based testing. You can write a suite of tests that will spin up your application and test it out, going from screen to screen, URL to URL and asserting all is as you would expect.

The project that I'm currently working on has a pretty comprehensive set of tests covering the use of the application. Each night as the clock strikes midnight a lonely computer in the West End of London whirrs into life and runs the full suite. It takes about 8 hours and at the end a report slips into your inbox letting you know of any failures.

Sounds brilliant right? How could someone not love this?#

Well a number of reasons. First of all, it takes 8 hours!!!! That's a long time; I'd rather learn what I broke today rather than tomorrow.

Also, and this is probably more significant, Coded UI tests are pretty flaky. Let me qualify that. For a test to be particularly useful it has to be quick, repeatable and reliable. As I've said, Coded UI tests are not quick.

By their very nature integration tests (of which Coded UI tests are a type) can never be entirely reliably repeatable. They test your app in it's entirety. So, for example, if a 3rd party service goes down for 5 minutes then you will get failed tests. You'll burn time investigating these false positives.

Further to that, Coded UI tests are repeatable, except when they're not. I've seen colleagues reduced to near tears by incredible sensitivity of Coded UI tests. Out of the box Coded UI tests appear to ship with the "Works on my machine" guarantee. It requires far more effort that you'd expect to come up with tests that can be reliably expected to pass. They will fail for surprising reasons. For instance, did you know that using the 2.x branch of jQuery won't work with Coded UI? Neither did I. I've lost track of the time that has been wasted running the same test in multiple different environments trying to identify what exactly is upsetting Coded UI about the environment this time.

It is sad but true that with Coded UI tests you can spend an enormous amount of time maintaining the test pack on a day to day basis. As infrastructure and project dependencies are upgraded you will sadly discover Coded UI has once again gone into the foetal position and has to tempted back to normal functioning by whispering sweet nothings in it's ear. ("It's not true that they've ended support for Windows XP" / "IE 6 will live forever" and so on)

Coded UI also appears to be badly supported by Microsoft. Documentation is pretty sparse and, as we'll come back to in a minute, Coded UI is sometimes broken or damaged by other products shipped by Microsoft. This makes it hard to have faith in Coded UI. Indeed, if you're thinking of automating your QA testing my advice would be "look into Selenium". Not because I've used it (I haven't) but those I've met who have used Selenium and Coded UI say Selenium wins hands down.

And yet, and yet...#

All of the above said, if you have a Coded UI test suite it can still pay dividends. Significant dividends. As I mentioned, my current project has a significant coverage of Coded UI tests. We've crawled over a lot of broken glass to put these together. But now they're there it is undeniably useful.

Every now and then we'll do a significant refactor of part of the application. For instance, we've entirely changed our persistence strategy in the app but been able to check the code in with a high degree of confidence gleaned from running our test suite using the refactored codebase.

Let me be clear: Coded UI tests can be useful.

The "runas" Problem#

Long preamble over, this post is about how to work around the latest issue Coded UI has thrown in our direction. I call it the "runas" problem. Our application is a Knockout / ASP.Net MVC web app built to be used in an intranet environment. By that I mean that identity is handled by Active Directory / Windows Authentication. When someone logs into our app we know who they are without them having to directly supply us with a username and password. No, by logging into their computer they have announced just who they are and Internet Explorer (for it is he) will pass along the credentials. (The app can be used with pretty much any browser but we're only mandated to support IE 9+.)

In order that we can test the app we have a number of test accounts set up in Active Directory. These test accounts have been assigned various roles (viewer / editor / administrator etc). Our tests are designed to run using these accounts in order that all scenarios can be adequately tested.

To achieve this lofty goal the following code (or something very like it) is executed as the first step in any Coded UI test:

string browserLocation = "C:\\Program Files\\Internet Explorer\\iexplore.exe";
string url = "http://localhost:12345/";
string username = "test.editor";
string domain = "theDomain";
var password = new SecureString();
foreach (char c in "test.editor.password")
{
password.AppendChar(c);
}
ApplicationUnderTest.Launch(browserLocation, null, url, username, password, domain);

What this does is fire up Internet Explorer as the supplied user of theDomain\test.editor, and navigate to the home page. With that as our starting place we could dependably then run a test as this test user. This was a solution not without quirks (on occasion Coded UI tests would "stutter" - repeating each keypress 3 times with calamitous effects). But generally, this worked.

Until that is either Visual Studio 2013 Update 3 or Internet Explorer 11 was installed. One of these (and it appears to be hotly contested) broke the ability to run the above code successfully. After these were installed running the above code resulted in the following error message:

"The application cannot be started. This could be due to one of the following reasons:

  1. Another instance of the application is already running and only one instance can be running at a time.
  2. The application started another process and has now stopped. You may need to launch the process directly.
  3. You do not have sufficient privileges for this application." File: C:\Program Files\Internet Explorer\iexplore.exe."

Lamentably, this was pretty much unresolvable and logging it with Microsoft yielded nothing helpful. This is what I mean about Coded UI being badly supported by Microsoft. Despite my best efforts to report this issue both to Connect and elsewhere and in the end nothing useful happened.

So what to do? I still have Coded UI tests, I still need to be able to run them. And crucially I need to be able to run them impersonating a different user. What to do indeed....

The hack#

workaround

After IE 11 / Visual Studio Update 3 / whatev's was installed I was left with a setup that allowed me to run Coded UI tests, but only

as the current user. On that basis I started looking into a little MVC jiggery pokery. All my controllers inherit from a single base controller. Inside there I placed the following extra override:

public abstract class BaseController : System.Web.Mvc.Controller
{
//...
protected override void OnAuthorization(AuthorizationContext filterContext)
{
#if DEBUG
if (filterContext.HttpContext.IsDebuggingEnabled)// Is compilation debug="true" set in the web.config?
{
var userToImpersonate = Session["UserToImpersonate"] as string;
if (!string.IsNullOrEmpty(userToImpersonate))
{
// userToImpersonate example: "[email protected]"
filterContext.HttpContext.User = new RolePrincipal(new WindowsIdentity(userToImpersonate));
}
}
#endif
base.OnAuthorization(filterContext);
}
//...
}

Each request will trigger this method as one of the first steps in the MVC pipeline. What it does is checks the Session for a user to impersonate. (Yes I'm as wary of Session as the next chap - but in this case it's the right tool for the job.) If a user has been specified then it replaces the current user with the Session user. From this point forwards the app is effectively running as that user. That's great!

In order that Coded UI can make use of this mechanism we need to introduce a "hook". This is going to look a bit hacky - bear with me. Inside Global.asax.cs we're going to add a Session_Start method:

protected void Session_Start(object sender, EventArgs eventArgs)
{
#if DEBUG
// If a user to impersonate has been supplied then add this user to the session
// Impersonation will happen in the OnAuthorization method of our base MVC controller
// Note, this is only allowed in debug mode - not in release mode
// This exists purely to support coded ui tests
if (Context.IsDebuggingEnabled) // Is compilation debug="true" set in the web.config?
{
var userToImpersonate = Request.QueryString["UserToImpersonate"] as string;
if (!string.IsNullOrEmpty(userToImpersonate))
{
Session.Add("UserToImpersonate", userToImpersonate);
}
}
#endif
}

For the first Request in a Session this checks the QueryString for a parameter called UserToImpersonate. If it's found then it's placed into Session. With this hook exposed we can now amend the first step that all our Coded UI tests follow:

// Various lines commented out as doesn't work with IE 11 - left as an example of how it could be done in the past
//string browserLocation = "C:\\Program Files\\Internet Explorer\\iexplore.exe";
string url = "http://localhost:12345/";
string username = "test.editor";
string domain = "theDomain.com";
//var password = new SecureString();
//foreach (char c in "test.editor.password")
//{
// password.AppendChar(c);
//}
//ApplicationUnderTest.Launch(browserLocation, null, url, username, password, domain);
// Suffixing url with UrlToImpersonate which will be picked up in Session_Start and used to impersonate
// in OnAuthorization in BaseController. Also no longer using ApplicationUnderTest.Launch; switched to
// BrowserWindow.Launch
// No longer used parameters: browserLocation, password
var userToImpersonate = username + "@" + domain; // eg "[email protected]"
var urlWithUser = url + "?UserToImpersonate=" + HttpUtility.UrlEncode(userToImpersonate);
var browser = BrowserWindow.Launch(urlWithUser, "-nomerge"); // "-nomerge" flag forces a new session

As you can see we actually need less when we're using this approach. We no longer need to directly specify the password or the browser location. And the user to impersonate is now passed in as the part of the initial URL used to launch the test.

Pay careful attention to the "-nomerge" flag that is passed in. This ensures that when another browser instance is opened a new session will be started. This is essential for "multi-user" tests that run tests for different users as part of the same test. It ensures that "test.editor" and "test.different.editor" can co-exist happily.

What do I think of the workaround?#

This approach works reliably and dependably. More so than the original approach which on occasion wouldn't work or would "stutter" keypresses. That's the good news.

The not so good news is that this approach is, in my view, a bit of hack. I want you to know that this isn't my ideal.

I really don't like having to change the actual system code to facilitate the impersonation requirement. Naturally we only ship the release and not the debug builds to Production so the "back door" that this approach provides will not exist in our Production builds. It will only be accessible in our development environments and on our Coded UI test server. But it feels oh so wrong that there is an effective potential back door in the system now. Well, only if the stars were to align in a really terrible (and admittedly rather unlikely) way. But still, you take my point. Caveat emptor and all that. This is something of a cutdown example to illustrate the point. If anyone else intends to use this then I'd suggest doing more to safeguard your approach. Implementing impersonation whitelists so "any" user cannot be impersonated would be a sensible precaution to start with.

Perhaps this is just one more reason that I'm not that enamoured of Coded UI. Once again it is useful but I've had to compromise more than I'd like to keep it's use. If anyone out there has a better solution I would love to hear from you.