Early Feedback - Testing Global ready Apps - using pseudo-localization

I have been talking, preaching and practicing on how to bring testing early on in development cycle to help improve time to market and help build baked in quality.

Early Testing of apps for global readiness is one of my top learning goal for some time. Use of pseudo-localization techniques to test global readiness of app has caught my fancy recently. My colleague Priyanka's(priyaag@thoughtworks.com) enthusiasm has only added to my desire to learn even stronger.

Recent tool support in open source world for pseudo-localization has only fueled my goal of learning.
Google's Pseudolocalization tool makes testing early much easier. I found it difficult to understand the usage of tool with the available resources.Priyanka and myself put effort to understand the usage of tool.With some effort we were able to learn this tool and we would like to share our understanding of the tool for wider community.

I am going to showcase usage of this tool, using a simple application that we built from using the sample application InternationalizedMortgageCalculator.java on Java internationalization trail

Before I go there, below are some links that have some excellent guidelines, definitions, testcases and studies around i18n,l10n and pseudo-localization testing that i found very insightful.

Excellent introduction to Pseudolocalization on wikipedia

Helen Kask's thesis gives in depth understanding of various aspect of internationalization and pseudolocalization, options available and some pros and cons of various approaches - Pseudolocalization Realization for Skype Software

Google open source blog have some simple but excellent example to understand Pseudolocalization to catch i18n errors

Some excellent checklists and testcases below -

Lingotek Internationalization and Pseudo Localization data sheet

Localization and Internationalization testing knowledge paper - excellent test case and approach here Localization and Internationalization testing approach and testcases

Learning to use Google Pseudolocalization tool with Sample app -

1 : Build the sample app

create a java project in eclipse with sample java swing program InternationalizedMortgageCalculator.java

along with the resource files supplied

• resources/Resources.properties
• resources/Resources_ar.properties
• resources/Resources_fr.properties 

screenshot of the project in eclipse

2 : Play around with application

Run the application by right clicking the InternationalizedMortgageCalculator.java and selecting "Run As" -> "Java Application"

Use various "locales" using "Locale" drop down menu and see how selecting different locales and regional language setting changes the UI text and direction (for e.g. selecting ar-AR will make the direction of text and order and location of Menu items from Right to Left) , and locale sensitive information such as Currency etc.

3: Create Pseudo-localized resource file

 Lets create a pseudo localized file for the default locale using Google "Pseudolocalization tool"

Step1 Download

Download the following three in a folder of your choice from http://code.google.com/p/pseudolocalization-tool/downloads/list

pseudolocalization-0.2-deps.jar

pseudolocalization-0.2.jar

pseudolocalizer (command line script for using the tool)

I have downloaded under $Home/pseudolocalizationtool

Step2 Configure

edit the "pseudolocalizer" shell script and change the DIR variable to point to the folder contatining the above three files. Save

Below is the screenshot for edited pseudolocalizer file as per my setting -

Step3 Run the tool

 Open a terminal window.  cd to your pseudolocalization tool folder.

> sh pseudolocalizer  <path to your default Resources.properties file in your eclipse project>

like this:

note: you can find the full path to your Resource.properties by right clicking "Resources.properties" in the eclipse project explorer and selecting "properties" and copying location from the resulting window.

Once you refresh the project in eclipse , you will find a new file created by tool "Resources_psaccent.properties".

The default variant is "psaccent". You will notice that "Resource_psaccent.properties" has

1. All the US-ASCII characters in original "Resource.properties" are accented

2. There will be brackets around every messages coming from your "Resources.properties" file

3. All the messages have been made longer by adding more characters and spaces

Step5 :Run the application with pseudolocalized resource

In Eclipse, Rename the "Resource.properties" file to "Resource_original.properties" and  "Resources_psaccent.properties" to "Resources.properties".

Run application from eclipse

You will see the application UI populated with pseudo-localized text.

Step5 :Run the application with fakebidi (using Right to left pseudolocalization)

Run the pseudolocalization tool with --variant flag set to "psbidi" to create a fakebidi resource file

Refresh eclipse project

Rename "Resource.properties" to "Resource_psaccent.properties"

Rename "Resource_psbidi.properties" to "Resource.properties"

Rerun the app and you will see the UI populated with "fakebidi"

Below is the screenshot of the same app -

Hope you find this blog useful from the perspective of getting started with "pseudolocalization" tool.

vodQA Bangalore

vodQA, the Thoughtworks testing conference is happening in Bangalore on 11 Feb 2012.The format seems to be quite interactive and 11 Feb looks quite an exciting day for those who see testing as the vital cog for IT and business agility.

The key thing is - there is no entry barrier as the conference participation is free.

I am participating in a panel discussion along with some of my esteemed colleagues who along with them bring plethora of experience on the topic of agile testing.The discussion should bring out paradigms, challenges , adoption patterns ,advancement in practice and toolsets etc.

There are lightning talks, interactive agile games and showcase of agile testing.

I am looking at the opportunity to exchange thoughts and experiences with participants.

Here is the link to event-page

http://testing.thoughtworks.com/events/agile-testing-teams-and-enterprises

Why Testing is perceived as least agile: What needs to change

A story about how testing failed the business

I was on an agile project. The test team started with great objective of keeping pace with test automation. Iteration 1 to 3 were quite uneventful, every test designed was automated. By this time Customer had first feedback from actual user. Entire UI design was changed, none of the tests were running, development team made changes as per the new UI requirement in a week, it took 2 weeks for automation team to change scripts to make these pass. By this time the test team had 2 weeks of automation backlog. Few more changes and backlog had increased to 6 weeks on a 12 week long project. Project team was getting nervous because of lack of regression testing, and pulled every tester in manual testing.
Later, some business users and developers were available to provide some additional bandwidth. But the automated tool was difficult to learn, proprietary in nature, looking at the scripts it was difficult to understand what the tests were trying to achieve as tests were programs. We couldn’t leverage skills of available resources. Business wanted to push through another release to preempt competition. Lack of automation meant delivery organization needed more time in manual testing effort.
Business was under dilemma- go with release date,may create loss of face or Don’t release give the advantage to competition on a platter.

This is not an isolated case nor is it going to be a one off occurrence in future. As long as current state of testing allows above situations occur, testing will be seen as non agile, may be party poopers or associated with waste.


Business and IT need to think testing strategically

If we see testing in the light of what business or organizations want to do, this perception might change. Testing needs to use business needs to improve efficiency in testing. There are some key business driver for testing as I see from my experience -

• Reduce or stabilize the cost of testing

• Improve release cycle to production. (Time to market, beat the competition)

• Reduce time spent in regression testing

• Reduce defect escape rate - Improve the quality of deployed application.

• Early Feedback of what application does and doesn’t do, allowing business to make informed decisions.


Testing should address Business and IT needs

Well, looking at the above business driver and putting my waste/efficiency hat, I think this is what roughly translates into requirement for testing –

• Testing must keep pace with development

• Test asset created must be able to run on multiple development and deployment platforms when any change happens.

• Business context should drive testing; business must be involved in testing.

• Test must be executable

• Test design and execution happens in real time

• Testing results reported in a way business understands

• Testing should not hinder change in business directions


Testing is a multi skill activity: Testing needs collaboration and innovation

I think I clearly understand the requirement now, however I see there are gaps, sometimes big gaps in the current state of tools and practices which will prevent me to achieve what I wanted to achieve –
• Key Business values and features are often not adequately tested

• Business is not involved in testing

• Test Automation tools lack the ability to write tests in language of business, The intent of tests are not clear.
• Converting test specification or intent of tests into executable tests often requires building a test framework from the ground up.

• Some automation Tools don’t support the programming language used by developers, hindering leveraging developer skills.

• Making changes in executable tests are non trivial activities. Test Automation Engineers spend most of their time maintaining tests. (Waste) rather than adding more tests (Value).

• Cost of tools and lack of flexibility of tools to support multiple programming languages means that organizations can’t leverage combined efforts of development and testing team.


Clearly, this means that testing should be far more collaborative than what it is now and requires far more collaborative effort in building tools and practices which leverages strength of individuals on the team and just not the testers, to make it effective and efficient.

Bringing Test Automation Early in Development lifecycle:Looking beyond Test Automation tool

While everyone agrees the benefits of early and continuous automated regression testing, few agree that test automation in early part of application development life cycle is such a good idea because of the effort and skill sets involved in making this happen. Having a test suite and infrastructure which allows deterministic and repeatable run is a key success criterion. Keeping test maintenance cost to a minimum under rapidly changing application is another challenge .One of the reasons of test automation not being successful early in application development lifecycle is not thinking beyond test automation tool. There are few basic problems where thinking beyond functional test automation tool will help and allow testing teams to do early test automation effectively. I have used few of the strategies listed below with relative success-

Dealing with changing functionality nightmare

– Right happy path tests, once stable include others

– Find a roadmap from customers/ BAs -It’s worthless to automate tests which fail before being useful. Knowing the functionality road map helps in picking up a test for automation at the right time.

Dealing with frequent application schema changes

– Baseline data and apply incremental changes to schema as schema changes

– In case database structure is governed by object models, Test should build the data through application layer, till there are few changes in object Model. Once object model that take a dump of your base lined database and apply incremental schema changes.

Dealing with Brittle UI

– Use a tool which can be integrated in an IDE being used by developers, so that developers can fix it when they are making changes.

– Don’t Hard code ObjectIDs, Maintain in a common object Repository.

– Don’t try automating, try API route, test all business rules. Be aware - don't take the API plunge lightly as it may require precious development resources and you might have to throw them later on. Do a trade off between no automation vs. automation through API carefully.

Dealing with in deterministic response from external application interfaces- Create stubs which return precooked responses based on some token in the request but the format of response is governed by interface definition.