Category: Uncategorized

GTP for BDD

Graphical Test Plan

I read a little about graphical test planning created by Hardeep Sharma and championed by David Bradley, both from Citrix. It’s a novel idea and sort of similar to the mind map test planning I have played around with. The difference is your not capturing features or various heuristics and test strategies in a mind map, you are mapping expected behavior only. Then you derive a test plan from the graphical understanding of the expected behavior of the system. I don’t know a lot about GTP, so this is a very watered down explanation. I won’t attempt to explain it, but you can read all about it:

Plan Business Driven Development with GTP

What interested me was the fact that I could abstract how we currently spec features into a GTP type model. I know the point of GTP is not to model features, but our specs model behavior and they happen to be captured in feature files. Its classic Behavior Driven Development (BDD) with Gherkin. We have a feature that defines some aspect of value that the system is expected to provide to users. In the feature we have various scenarios that describe the expected behaviors of the feature. Scenarios have steps that define pre-condition, action, and expectations (PAE) or in Gherkin, Given-When-Then (GWT) that define how a user would execute the scenario. We also have feature backgrounds which is a feature wide pre-condition that is shared by all scenarios in the feature.
I said we use Gherkin, but our new test runner transcends just GWT. We can define PAE in plain English without the GWT constraints, we can select the terms to describe PAE instead of being forced to use GWT which sometimes causes us to jump through hoops to force the GWT wording to sound correct. 

GTP Diagram

If we applied something like GTP we would model the scenarios, but there would be more hierarchy before we define the executable scenarios. We currently use tagging to group similar scenarios that exercise a specific subset of a feature’s scenarios. This allows us to provide faster feedback by running checks for just a subset instead of the entire feature when we are only concerned with changes to the subset. In a GTP’ish model the left most portion of the diagram would hold generalized behavior specs, similar to how we use tagging, and as we go to the right the behavior becomes more granular until we hit a demarcation point for executable scenarios that can then be expressed in a linked test case diagram (TCD). In the GTP there are ways to capture meta data like related requirement/ticket ID for traceability back to requirements. Also, meta for demarcation point (can’t think of better name) to link to the TCD or feature file that further defines it.

Test Case Diagram

The test case diagram would define various scenarios that define the behavior of the demarcation points in the GTP. The TCD diagram would also include background preconditions and the steps to execute the scenario. At this point it feels like this is an extra step. We have to write the TCD in a feature file so diagramming it is creating a redundant document that has to be maintained.
In the TCD there are shapes for behavior, preconditions, steps, and expectations. I think there should be additional shapes or meta to express tags because this is important in how we categorize and control running of scenarios. It may help if there is also meta to link back to the GTP that the TCD is derived from so we can flow back and forth between the diagrams. Meta in the TCD is important because it gives us the ability extract understanding outside of just the test plan and design. We could have shapes, meta descriptions and links to
  • execute automated checks
  • open a manual exploratory test tool
  • view current test state (pass/fail)
  • view historical data (how many times has this step failed, when was the last failure of this scenario…)
  • view flake analysis or score
  • view delivery pipeline related to an execution
  • view team members responsible for plan, develop, test and release
  • view related requirement or ticket
  • much more…

Since we also define manual tests by just tagging features or scenarios with a manual tag or creating exploratory test based feature files, we could do this for both automated checks and manual tests.

GTP-BDD Binding

To get rid of the TCD redundancy we could generate the feature file from the diagram or vice-versa. Being able to bind GTP to BDD would make GTP more valuable to me.
We would need an abstract object graph that could be used to generate both the diagram and the feature file (Excel spread sheet, HTML page or whatever else). We are almost here, we have a tool that can generate feature files from persisted objects and vice versa. We would just have to figure out how to generate the diagram and express it as an interactive UI and not just a static picture.
What we have been struggling with is the ability to manually edit feature files and keep that in sync with the persisted objects. With a centralized UI this is easy because everyone uses the UI to update the objects. When people are updating features files from a source code repository we have to worry about merge conflicts (yuck) and if we consider the feature file or the persisted object as the source of truth. So, we may have to reduce flexibility and force everyone to use the UI only. Everyone would have to have discipline and not touch the feature files even though we have nice tools built into our IDE to help write and manage them. The tool would have to detect when someone has violated the policy and so on…I digress.

Conclusion

With a graphical UI modeled on GTP/TCD to manage BDD we can provide an arguably simpler way to visualize tests and provide the ability to drill down to see different aspects of test plans and designs and their related current and historical execution. With 2-way binding from diagram to feature file we have a new way to manage our executable specifications. This model could provide a powerful tool to not only aide test planning, but test management as a whole. The end result would hopefully be a better understanding for the team, increased flow in delivery pipeline, enhanced feedback, and more value to the customer and the business.
Now lets ask Google if something like this already exists so I don’t have to add it to my ever increasing backlog of things I want to build. Thanks to Hardeep Sharma, David Bradley, and Citrix for sharing GTP.

What is Object Oriented Programming?

Until today I never contemplated what object orientated programming means. I have thought about OOP in the context of writing object oriented code and figuring out how to be better at OOP, but I never really studied its original design and intent. I was listening to a .Net Rocks podcast on the subject of lean functional programming and they had a discussion that led to Dr. Alan Kay’s definition of OOP. Dr. Kay coined the phrase object oriented programming during his work developing the SmallTalk language. Even though he is credited with the term OOP, what OOP became is not the same as he envisioned it.

I did a little research and landed on this email thread between Stefan Ram and Dr. Kay.

http://www.purl.org/stefan_ram/pub/doc_kay_oop_en

Dr. Kay credits his thoughts on OOP to his background in biology and mathematics. He says,

I thought of objects being like biological cells and/or individual computers on a network, only able to communicate with messages (so messaging came at the very beginning — it took a while to see how to do messaging in a programming language efficiently enough to be useful). Dr. Alan Kay, July 17, 2003

Even polymorphism wasn’t a part of the his original thoughts on OOP. He proposed another term to describe behavior in related objects, “genericity.”

He ends the thread by giving his definition of OOP,

OOP to me means only messaging, local retention and protection and hiding of state-process, and extreme late-binding of all things. Dr. Alan Kay, July 23, 2003

Doesn’t this sounds like functional programming?

Even though this is years after Dr. Kay’s original OOP work he may have had changes in thought over the years. Yet, I still believe that his intent is still true here. So it seems that through the years, OOP may have been forked or hijacked. Some argue that we are paying for it today. They say that OOP did not stay true to the OOP of Dr. Kay and his peers in the early days of OOP (am I spamming OOP here?).

Now, I have another reason to learn F#. With the little I know now, I have already started trying to remove the ill affects of mutable state and use message passing to communicate across layers. I know there is so much more to learn and I have been doing this a long time. I look forward to digging into F# maybe even Erlang to see how it can influence my OOP.

Mars Drone

I am proud to announce that I will be finally launching my Mars Drone today. I have been building this project for 5 years. The drone is very small like a drone you would buy in a local store. It is heavily shielded and runs on a proprietary propulsion and energy system. The drone finally passed the final test of its guidance and communications system last week when it returned successfully from a lunar orbit. If all goes well, I will be able to publish pictures of Mars in about nine months.

Here’s to dreaming!

How to install a DLL to the GAC on Windows Server 2012 using only PowerShell (without having to install SDK or Visual Studio)

Easy way to install DLL to GAC without SDK or VS.

Cameron Dwyer

Prior to Windows Server 2012 I had been use to installing DLL files in the Windows Global Assembly Cache (GAC) by either opening the Windows/Assembly folder in Explorer and simply dragging and dropping the file, or by using GacUtil.exe

With Windows Server 2012 unfortunately it’s not quite so easy. Being able to simply open the GAC in Explorer and drag/drop is gone. Also GacUtil.exe is not present on the server by default.

PowerShell to the rescue. Here’s how to register a DLL called “MyDLL.dll” to the GAC (and also how to remove it.

For this example assume we have the “MyDLL.dll” file stored at c:tempMyDLL.dll

To add a DLL to the GAC

1. Run the SharePoint PowerShell console as Administrator

2. Enter the following PowerShell

To remove a DLL from the GAC

1. Run the SharePoint PowerShell console as Administrator

2. Enter the following PowerShell

To view information for a…

View original post 28 more words

C# MEF BrowserFactory for Browser Based Testing

In my browser based test framework I use MEF to help abstract the concept of a browser. The reason I do this is so I am not tied to a specific browser driver framework (e.g. WebDriver, WatiN, System.Net.WebClient). This allows me to change drivers without having to touch my test code. Here’s how I do it.

Browser Interface

First I created an interface that represents a browser. I used a mixture of interfaces from WebDriver and WatiN.

namespace CharlesBryant.TestPipe.Interfaces
{
 using System;
 using System.Collections.Generic;
 using System.Collections.ObjectModel;
 using CharlesBryant.TestPipe.Browser;
 using CharlesBryant.TestPipe.Enums;
public interface IBrowser
 {
 IBrowserSearchContext BrowserSearchContext { get; }
BrowserTypeEnum BrowserType { get; }
string CurrentWindowHandle { get; }
string PageSource { get; }
string Title { get; }
string Url { get; }
ReadOnlyCollection WindowHandles { get; }
IElement ActiveElement();
void Close();
void DeleteAllCookies();
void DeleteCookieNamed(string name);
Dictionary<string, string> GetAllCookies();
bool HasUrl(string pageUrl);
void LoadBrowser(BrowserTypeEnum browser, BrowserConfiguration configuration = null);
void Open(string url, uint timeoutInSeconds = 0);
void Quit();
void Refresh();
void SendBrowserKeys(string keys);
void TakeScreenshot(string screenshotPath);
void AddCookie(string key, string value, string path = "/", string domain = null, DateTime? expiry = null);
}
}

Pretty basic stuff although BrowserSearchContext took some thought to get it working. Basically, this abstraction provides the facility to search for elements. A lot of the concepts here are borrowed from WebDriver and WaitN and are just a way to be able to wrap there functionality and use it without being directly dependent on them. To use this you have to change your tests from directly using a browser driver to using this abstraction. At the start of your tests you use the BrowserFactory to get the specific implementation of this interface that you want to test with.

Browser Factory

Then I created a BrowserFactory that uses MEF to load browsers that implement the browser interface. When I need to use a browser I call Create in the BrowserFactory to get the browser driver I want to test with. To make this happen I have to actually create wrappers around the browser drivers I want available. One caveat about MEF is that it needs to be able to find your extensions so you have to tell it where to find them. To make the browsers available to the factory I added an MEF class attribute, [Export(typeof(IBrowser))] to my browser implementations. Then I add a post build event to the browser implementation projects to copy their DLL to a central folder:

copy $(TargetPath) $(SolutionDir)\Plugins\Browsers\$(TargetFileName)

Then I added a appConfig key with a value that points to this directory to the config of my clients that use the BrowserFactory. Now I can reference this config value to tell MEF where to load browsers from. Below is sort of how I use the factory with MEF.

namespace CharlesBryant.TestPipe.Browser
{
 using System;
 using System.ComponentModel.Composition;
 using System.ComponentModel.Composition.Hosting;
 using System.Configuration;
 using System.IO;
 using System.Reflection;
 using CharlesBryant.TestPipe.Enums;
 using CharlesBryant.TestPipe.Interfaces; 

 public class BrowserFactory
 {
 [Import(typeof(IBrowser))]
 private IBrowser browser; 

 public static IBrowser Create(BrowserTypeEnum browserType)
 {
 BrowserFactory factory = new BrowserFactory();
 return factory.Compose(browserType);
 } 

 private IBrowser Compose(BrowserTypeEnum browserType)
 {
 this.browser = null; 

 try
 {
 AggregateCatalog aggregateCatalogue = new AggregateCatalog();
 aggregateCatalogue.Catalogs.Add(new DirectoryCatalog(ConfigurationManager.AppSettings["browser.plugins"])); 
 CompositionContainer container = new CompositionContainer(aggregateCatalogue);
 container.ComposeParts(this);
 }
 catch (FileNotFoundException)
 {
 //Log
 }
 catch (CompositionException)
 {
 //Log;
 } 

 this.browser.LoadBrowser(browserType);
 return this.browser;
 }

namespace CharlesBryant.TestPipe.Enums
{
public enum BrowserTypeEnum
 {
 None,
 IE,
 Chrome,
 FireFox,
 Safari,
 Headless,
 Remote,
 Other
 }
}
 }
}

Conclusion

Well that’s the gist of it. I have untethered my tests from browser driver frameworks. This is not fully tested across a broad range of scenarios so there may be issues, but so far its doing OK for me.

The examples above are not production code, use at your own risk.