Tagged: webdriver
Video Recording C# WebDriver Tests in TestPipe
The title is a little misleading because you can use the technique below to do a screen capture of anything happening on the screen and not just WebDriver tests. Yet, TestPipe does use C# WebDriver out the box so we will be recording these types of tests.
So, we want to add video recording tests to TestPipe. At first I thought this would be very difficult, but after finding Microsoft Expression Encoder SDK it became a lot easier. I was even able to find other people that have used this SDK which made a decision to move forward with this a little easier to take on.
First, I read the Working with Screen Capture section of the Overview of the Expression Encoder SDK. From this I learned that I needed to create an instance of ScreenCaptureJob. The question is, where do I create it?
In TestPipe we have a ScenarioSession class that holds state of various things while a test scenario runs and it makes sense to expose this new functionality there because we want to be able to control video recording within the context of individual test scenarios. Do we add a new property on the session or should it be a new property on the IBrowser interface. We already have a TakeScreenshot method on IBrowser. Yet, I don’t think it is a good fit on the browser interface because there is a bit of setup that needs to take place for ScreenCaptureJob that is out of scope for a browser and I don’t want to muddy up the API more than it already is.
When we setup a scenario we want to allow setup of the ScreenCaptureJob based on configuration for a feature and/or a scenario. We define features and scenarios in a text file, currently using Gherkin, and we store data used in feature and scenario tests in a JSON file. So, we have to configure video recording in the Gherkin, JSON or both.
Do we keep all recordings or only failing recordings? What if we want to keep only failing, but from time to time we need non-failing recordings for some reason? Do we overwrite old recordings or store in unique folders or filenames?
To trigger the recording we could use tags. If an @Video tag is present on the scenario or feature, record the scenario(s) and only keep the recording if the scenario fails. If the @Debug tag is present on the Feature or the Scenario, keep the recordings even if they don’t fail.
We can create a unique folder for the recordings so that we can store videos of multiple runs of the same scenario. We may want to think about how we clean these up, but we may have enough file clean up in other processes. We will just have to watch hard drive space in production use.
So, we have a strategy to automatically configure recording. Now, we have to implement it in a way that also allows manual configuration just in case we want to hard wire video recording in a test.
So, I found our seam to make the changes for video recording. In our RunnerBase class we have methods to setup and teardown a scenario. It is there that we will make the change to configure, start, stop, and delete video recordings.
Now to implement. First I download the encoder from http://www.microsoft.com/en-us/download/details.aspx?id=27870. This will have to be installed on every server that will run tests so I create a Powershell script to install it. It would be nice to also do a Chocolatey package, but that is overkill for me because I am not using Chocolatey on my servers right not. You can create your own automated installer by extracting the setup file from the download then creating a Powershell script to run
setup.exe -q
to quietly install. I believe you can use the -x parameter to uninstall, but I didn’t test this yet. (Assuming msiexe command line options are used https://msdn.microsoft.com/en-us/library/aa367988(v=vs.85).aspx)
With the encoder installed we have access to the DLLs that we need to work with. In VisualStudio I add a reference to the extensions for Microsoft.Expression.Encoder, Microsoft.Expression.Encoder.Api2, Microsoft.Expression.Encoder.Types, and Microsoft.Expression.Encoder.Utilities. Not sure if I need all these, but they were added by the installer so I will keep them for now.
From here I can add a using
using Microsoft.Expression.Encoder.ScreenCapture;
and implement recording based on the sample code, updating to fit TestPipe standards.
One caveat is the encoder outputs some kind of Microsoft proprietary video format xesc. I thought about collecting all the videos that are kept at the end of a test run and run some kind of parallel task to convert them to a more portable format. In the end, I just left it alone. This is a new feature and only my team will be looking at the videos and everyone has Windows Media Player that can play the format.
I won’t write more on implementation details because I am boring myself, but if you want to check it out you can view it on GitHub (RunnerBase is where we use the recorder and you should be able to figure out the rest). One interesting twist is we implemented Expression Encoder behind an interface so that it isn’t requirement to use TestPipe. If we didn’t do this, you wouldn’t be able to build or use TestPipe without first installing the dependent encoder.
So, TestPipe comes out the box with a dummy implementation of the interface that won’t actually do the recordings. If you want to capture actual recording you can use the TestPipe.MSVideoRecorder plug-in or implement the IVideoRecorder interface on another screen capture program to enable video recording of tests. Right now TestPipe.MSVideoRecorder, is included in the TestPipe solution, but it is not set to build automatically. When we make changes we set it to build and manually move the binary to the folder we have configured to hold the video recorder plug-ins. Eventually, we will move it to a separate repository and create a NuGet package, but I’m tired.
References
Selenium WebDriver Custom Table Element
One thing I like to do is to create wrappers that simplify usage of complex APIs. I also like wrappers that allow me to add functionality to an API. And when I can use a wrapper to get rid of duplicate code, I’m down right ecstatic.
Well today I’d like to highlight a little wrapper magic that helps simplify testing HTML tables. The approach isn’t new, but it was introduced in TestPipe by a junior developer with no prior test automation experience (very smart guy).
First some explanation before code as it may be a little hard to understand. TestPipe actually provides a wrapper around Selenium Web Driver. This includes wrappers around the Driver, SearchContext, and Elements. TestPipe has a BaseControl that is basically a wrapper around WebDriver’s IWebElement. This not only allows us to bend WebDriver to our will, the biggest benefit, but it also gives us the ability to swap browser drivers without changing our test code. So, we could opt to use WatiN or some custom System.Net and helpers and not have to do much, if anything, in terms of adjusting our test code. (Side Note: this isn’t a common use case, almost as useless as wrapping the database just so you can change the database later…doesn’t happen often, if ever, but its nice that its there just in case)
GridControl
Like I said, we have a BaseControl. To provide some nice custom functionality for working with tables we will extend BaseControl.
public class GridControl : BaseControl
{
public GridControl(IBrowser browser)
: base(browser, null, null)
{
}
//...Grid methods
}
Breaking this down, we create a new class, GridControl, that inherits BaseControl. In the constructor we inject an IBrowser, which is the interface implemented by our WebDriver wrapper. So, outside of TestPipe, this would be like injecting IWebDriver. The BaseControl handles state for Driver and exposes it to our GridControl class through a public property. This same concept can be used to wrap text boxes, drop down lists, radio button…you get the idea.
GetCell
I won’t show all of the code, but it’s on GitHub for your viewing pleasure. Instead, I will break down a method that provide some functionality to help finding cells in tables. GetCell, this method provides a simple way to get any cell in a table by the row and column.
public IControl GetCell(int row, int column)
{
StringBuilder xpathCell = new StringBuilder();
xpathCell.Append("//*[@id='");
xpathCell.Append(this.SelectId.ToString());
xpathCell.Append("']/tbody/tr[");
xpathCell.Append(row);
xpathCell.Append("]/td[");
xpathCell.Append(column);
xpathCell.Append("]");
Select selector = new Select(FindByEnum.XPath, xpathCell.ToString());
IControl control = this.Find(selector);
return control;
}
This isn’t the actual TestPipe code, but enough to get the gist of how this is handled in TestPipe. We are building up an XPath query to find the tr at the specified row index and the td at the specified column index. Then we pass the XPath to a Selector class that is used in the Find method to return an IControl. IControl is the interface for the wrapper around WebDriver’s IWebElement. The Find method uses our wrapper around WebDrivers ISearchContext to find the element…who’s on first?
One problem with this approach is the string “/tbody/tr”. What if your table doesn’t have a tbody? Exactly, it won’t find the cell. This bug was found by another junior developer that recently joined the project, another very smart guy. Anyway, this is a problem we are solving as we speak and it looks like we will default to “/tr” and allow for injection of a different structure like “/tbody/tr” as a method argument. Alternatively, we could search for “/tr” then if not found search for “/tbody/tr”. We may do both search for the 2 structures and allow injection. These solutions are pretty messy, but it’s better than having to write this code every time you want to find a table cell. The point is we are encapsulating this messy code so we don’t have to look at it and getting a cell is as simple as passing the row and column we want to get from the table.
Usage in Tests
In our page object class we could use this by assigning GridControl to a property that represents a table on our page. (If you aren’t using page objects in your automated browser tests, get with the times)
//In our Page Object class we have this property
public GridControl MyGrid
{
get
{
return new GridControl(this.Browser);
}
}
Then it is pretty easy to use it in tests.
//Here we use myPage, an instance of our Page Object,
//to access MyGrid to get the cell we want to test
string cell = myPage.MyGrid.GetCell(2,5).Text;
Assert.AreEqual("hello", cell);
Boom! A ton of evil duplicate code eradicated and some sweet functionality added on top of WebDriver.
Conclusion
The point here is that you should try to find ways to wrap complex APIs to find simpler uses of the API, cut down on duplication, and extend the functionality provided by the API. This is especially true in test automation where duplication causes maintenance nightmares.
Validating Tab Order with WebDriver
I had a spec that defined the tab order on a form. Starting with the default form field the user will be able to press the tab key to move the cursor to the next form field. Tabbing through the fields will follow a specific order. I couldn’t find much on Google or Bing to help automate this with WebDriver, maybe I’m loosing my search skills.
Below is code to implement this with WebDriver. In production I use a SpecFlow Table instead of an array to hold the expected tab order and I have a custom wrapper around WebDriver so much of this code is hidden from test code. Below is the untested gist of my production implementation. Since all of my elements have IDs, and your’s should too, we are simply validating that the active element has the ID of the current element in the array iteration.
- If the element doesn’t have an ID, fail the test.
- If the element ID doesn’t match the expected ID, fail the test.
- If the ID matches, tab to the next element and loop.
public void TestTabOrder()
{
//Code to open the page elided.
....
//This is the expected tab order. The strings are element IDs so the test assumes all of your elements have IDs.
string[] orderedElementIds = new string[] { "FirstControl", "SecondControl", "NextControl" };
foreach (var item in orderedElementIds)
{
string elementId = item;
//Get the current active element, element with focus.
IWebElement activeElement = webDriver.SwitchTo().ActiveElement();
//Get the id of the active element
string id = activeElement.GetAttribute("id");
//If the active element doesn't have an id, fail the test because all of our elements have IDs.
if (string.IsNullOrWhiteSpace(id))
{
throw new AssertionException("Element does not have expected ID: " + elementId);
}
//If the active element doesn't match the current ID in our orderedElementIds array, fail the test.
if (elementId != id)
{
throw new AssertionException("Element: " + elementId + " does not have focus.");
}
//Tab to the next element.
activeElement.SendKeys(Keys.Tab);
}
}
You don’t have to assert anything as the exceptions will fail the test (using MSTest AssertionException), hence no exception equals passing test. You get a bonus assert with this test in that it also verifies that you have a certain element with default focus (the first element in the array).
I am sure there is a better way to do this, but it works. Hope this helps someone as it wasn’t something well publicized.
Selenium WebDriver File Download Testing
Not easy!
When you click a download button with the standard browser configuration you are asked for a location to download the file. Let’s explore the Google Encyclopedia of copy and paste code to see how we can solve this.
The main solution seems to be setting the browser profile to automatically download without asking for a location. Below I have a little info on what I found out about Firefox, Chrome and IE. I didn’t do a deep dive. Right now I am spiking a solution to see if this will need to be a manual test.
Firefox
From our friends at StackOverflow this posts advocates manually setting the Firefox profile then changing to the profile in Selenium (http://stackoverflow.com/questions/14645877/how-to-test-downloading-files-in-selenium2-using-java-and-then-check-the-downloa)
In firefox also you can do the same thing.
Create one firefox profile
Change the firefox download setting as it should save files without asking about location to save
Launch the automation using that profile.
FirefoxProfile profile = new FirefoxProfile(profileDir);
driver=new FirefoxDriver(profile);
Another little gem from the same site says I can do one better by configuring the profile in code (http://stackoverflow.com/questions/16746707/how-to-download-any-file-and-save-it-to-the-desired-location-using-selenium-webd):
firefoxProfile.setPreference(“browser.helperApps.neverAsk.saveToDisk”,”text/csv”);
And this post brings it home and puts it all together (http://stackoverflow.com/questions/1176348/access-to-file-download-dialog-in-firefox)
FirefoxProfile firefoxProfile = new FirefoxProfile();
firefoxProfile.setPreference(“browser.download.folderList”,2);
firefoxProfile.setPreference(“browser.download.manager.showWhenStarting”,false);
firefoxProfile.setPreference(“browser.download.dir”,”c:\\downloads”);
firefoxProfile.setPreference(“browser.helperApps.neverAsk.saveToDisk”,”text/csv”);
WebDriver driver = new FirefoxDriver(firefoxProfile);//new RemoteWebDriver(new URL(“http://localhost:4444/wd/hub”), capability);
driver.navigate().to(“http://www.myfile.com/hey.csv”);
Chrome
There are instructions on achieving the same results as Firefox, but you have to jump threw a few minor hoops to get there (http://stackoverflow.com/questions/15824996/how-to-set-chrome-preferences-using-selenium-webdriver-net-binding).
IE
There seems to be ways to do this in WatiN in browsers below IE9, but for current IE browsers its just plain ugly (http://stackoverflow.com/questions/7500339/how-to-test-file-download-with-watin-ie9/8532222#8532222). I guess I could use one of the GUI Automation tools (see GUI Automation post), but is it really worth all that?
I haven’t tried these at home, so I am naively assuming that they work. Anyway after you have successfully downloaded the file, you can use C# System.IO to inspect the file format, size, file name, content…you get the picture.
SpecFlow Ambiguous Step Definitions
It’s been a long time since I posted anything. I have a ton of material to post, just been too busy or lazy to post it.
Anyway, here is the problem. I use SpecFlow, Selenium WebDriver, and the Page Object Model pattern to implement UI tests. I want to scope my SpecFlow Step Definitions and I ran into this link that made me think twice about doing it. https://github.com/cucumber/cucumber/wiki/Feature-Coupled-Step-Definitions-%28Antipattern%29
The basic premise is that you shouldn’t tie your step definitions to features.
Feature-coupled step definitions are step definitions that can’t be used across features or scenarios. This is evil because it may lead to an explosion of step definitions, code duplication and high maintenance costs.
I can agree with this, but there should be a way to tie a step to a context. The best example of what I mean is when a step is only relevant to a page or control when doing UI tests. When you have a generic step definition, but the implementation can be specific to a page or control it makes sense to be able to scope the step to the page or control. For example, if we take the scenario from the wiki page above
Scenario: add description
Given I have a CV and I’m on the edit description page
And I fill in “Description” with “Cucumber BDD tool”
When I press Save
Then I should see “Cucumber BDD tool” under “Descriptions”
(Note: Save is a regular expression on the wiki, but I rely on page object models so it isn’t necessary to pass the value to the step method as my actions are explicit.)
The “When I press Save” step is going to call a method in a page object to actually press the Save button. The step will use a specific page object to execute the step and this generic definition does not provide any context to say which page object to use. If I could scope the step definitions and implementations to a particular page or control, I could have various implementations to target various page objects to drive the page or control under test.
With this we are not coupling by feature, but by page or control. Is this bad or another anti-pattern? Time will tell, but I have had the hardest time trying to name steps with context identifiers to try to get around the problem of step definitions having a global scope in SpecFlow. If I had another scenario that used the “When I press Save” definition, but is implemented with a different page object we run into ambiguity issues because SpecFlow doesn’t know which implementation to use. Without a scoping mechanism I have to add context to the step definitions. Our simple definition would become, “When I press Save on the CV Edit Description page”. This usually makes defining steps and reading them a lot harder than it should be because I have to use more words.
As a general practice, I normally scope my features and scenarios with a tag indicating the page or control under test and this could easily be used in the step definitions to couple step implementations to specific pages and controls. With SpecFlow we can use a feature called Scoped bindings to achieve page/control scoped step definitions.
The Scope Attribute can be used to restrict the execution of step definitions by features, scenario, or by tag. Since scoping by feature is an anti-pattern we won’t use that one. The scenario is a viable restriction, but I believe tag will provide the most flexibility as we can restrict multiple scenarios across various features without limiting the step to a particular feature or scenario. Although, there is the problem of tags only being applied at the scenario level. We can not tag a scenario step in SpecFlow, i.e. tag the Given, When and Then separately. I am not sure if this would be necessary. I have to get more specs written with scoped bindings to see what troubles I run into.
You can look at the Scoped bindings link for usage, but in our scenario above we could use this technique by tagging the scenario with the page name:
@CVEditDescription
Scenario: add description
Given I have a CV and I’m on the edit description page
And I fill in “Description” with “Cucumber BDD tool”
When I press Save
Then I should see “Cucumber BDD tool” under “Descriptions”
Then the “When I press Save” step can be scoped to the CV Edit Description page like so:
[When (@”I press Save”, Scope(Tag = “CVEditDescription”)]
public void WhenIPressSave()
{
//Call Save on the page object
cvEditDescriptionPage.Save();
}
We also get the added benefit of being able to run tests just for this page (tag). So far I like it. How do you solve ambiguity issues in SpecFlow?
Decoupled Logic 