#3 Don’t become complacent in testing when a change is simple.

The Heartbleed bug was caused by a somewhat simple change that was not properly tested before it was released to the public. Maybe that is a overreaching generalization, but the fact is, the change and the eventual fix to Heartbleed were trivial, but incredibly damaging. I have found time and time again that many bugs are introduced through simple changes that are not properly tested before releasing them because they are so simple. I’m just inverting some logic, doing a simple refactor, updating how a variable is updated, adding a column to a query…I can create a long list of bugs that jump in my mind that were thought to be simple changes, but leaked risk and chaos into a release.

Quality Starts with Development

When I say test, I’m not talking about QA testing, I’m talking about developer testing. Quality is a team sport and the developers have the ball most of the game. The fact is developers have the largest share in insuring quality. There are usually more developers than any other role on the product team. The developers usually eat up most of the clock in the release cycle.

QA just got stuck with quality in their title because developers where a custom to tossing their code across the fence for others to worry about it working or not. QA was enlisted to sit on the other side of the fence and catch the bugs. OK, that is not the true story of how the QA role came to be, but that’s how I imagine it. I have seen this happen so many times: developers deliver code at last the last minute to QA, QA has to rush to test it, a bug is found that could have easily been caught with a more thorough inspection of quality by the developer, then QA has to test it again and pray that the developer tested it as they deal with their backed up schedule with no time to thoroughly test. I don’t look at QA as quality managers, but risk managers. They measure, analyze, and report the risk of a release. They don’t build quality into the product developers do. They don’t even insure that the team ships a quality product. They only point out risks in a release version. It’s up to the team to insure a certain level of quality is maintained and the analysis by QA is a part of defining what level of quality is expected. In the end, the quality buck stops with development, but the team insures quality of the product. Yet, if something breaks in the code, its not QA or the BA that will code the fix.

Test It

Simple changes from the database to the UI have been found to be culprits of pesky, hard to find bugs. The one thing they all have in common is that they were not tested or the tests were not good enough to cover the change. I am not going to say that “every change has to have a unit test…roar!”, but you need to find a way to test the fix that is better than visual inspection of an awesomely simple code change.

To me the best test is from the perspective of the user. In the end its the user who has to suffer from crappy code. So, if you are hacking on a web app, open a browser and test your change. Developing a service, open Fiddler or WireShark and send a request to exercise your change. If you are into it, go ahead and write a unit test. Learn about testing, just like you learn about new technologies to make your coding life easier.

Developing is more than coming up with elegant algorithms and simple solutions to complex problems. Do what you have to in order to find a meaningful way to exercise your changes. Don’t stop with one test, find more than one way to test your change. Your goal should be to commit the code better than you checked it out. If not, your QA will waste time and money rejecting your change or worse your users will reject your change and you will risk the reputation and brand of your company on something you thought was simple.

Conclusion

Well, if you have been a developer long, you have heard most of this before. This was more of a rant against myself for being asleep at the wheel on a simple change, I am my biggest critic. I believe all good developers share my sentiments in some way. Yet, simple changes have a habit of biting us because of our complacency. To validate our assumptions about a change, no matter how simple, we must try to test the change through the UI or public API. Even a simple smoke test to verify the UI still works is better than no test at all. Running a stored procedure after a simple column change is better than just checking it in because it was so easy to add that misspelled column name. You can take the time now to test or later, but later may cost you and your team in the end.

As a Software Developer in Test, I will ask a developer how they tested their change just to measure their due diligence. If I don’t hear something about the UI or API, unit test or some kind of substantive test, I have an urge to release the hounds. When I release the hounds, the cats run up a tree. When the cats run up a tree they scare the birds and they poop on your mom’s car. Don’t get your mom’s car pooped on 🙂

View more tips.

When I am writing and maintaining large functional UI tests I often realize somethings that would make my life easier. I decided to write this series of posts to describe some of the tips I have for myself in hopes that they prove to be helpful to someone else. What are some of your tips?