Monday, May 5, 2008

Bringing Test Automation Early in Development lifecycle:Looking beyond Test Automation tool

While everyone agrees the benefits of early and continuous automated regression testing, few agree that test automation in early part of application development life cycle is such a good idea because of the effort and skill sets involved in making this happen. Having a test suite and infrastructure which allows deterministic and repeatable run is a key success criterion. Keeping test maintenance cost to a minimum under rapidly changing application is another challenge .One of the reasons of test automation not being successful early in application development lifecycle is not thinking beyond test automation tool. There are few basic problems where thinking beyond functional test automation tool will help and allow testing teams to do early test automation effectively. I have used few of the strategies listed below with relative success-

Dealing with changing functionality nightmare

– Right happy path tests, once stable include others

– Find a roadmap from customers/ BAs -It’s worthless to automate tests which fail before being useful. Knowing the functionality road map helps in picking up a test for automation at the right time.

Dealing with frequent application schema changes

– Baseline data and apply incremental changes to schema as schema changes

– In case database structure is governed by object models, Test should build the data through application layer, till there are few changes in object Model. Once object model that take a dump of your base lined database and apply incremental schema changes.

Dealing with Brittle UI

– Use a tool which can be integrated in an IDE being used by developers, so that developers can fix it when they are making changes.

– Don’t Hard code ObjectIDs, Maintain in a common object Repository.

– Don’t try automating, try API route, test all business rules. Be aware - don't take the API plunge lightly as it may require precious development resources and you might have to throw them later on. Do a trade off between no automation vs. automation through API carefully.

Dealing with in deterministic response from external application interfaces- Create stubs which return precooked responses based on some token in the request but the format of response is governed by interface definition.