How can kung fu stop something that stops kung fu?

Most software were developed by highly skilled programmers. But still none of them are bug free and many of them are very buggy.

“If builders built buildings the way programmers write programs, the first woodpecker that came along would destroy civilization.” – Jerry Weinberg

Software is a very volatile product, so it’s practically impossible to build bug free software even by the best of the programmers. So there exist testers to test the product. Testers look into a product with different perspective to inform management about the health of the product, thus helping management to take decisions better and faster. Most releases will go smooth and there will be failures in between (or vice versa 😉 ). Everything looks awesome until something goes wrong, then everything comes under the scanner. There will be a high-profile meeting to discover that THE problem was testing not done properly and THE reason was lack of time and THE solution is automation. 🙂

Now some testers who had talked about automation earlier will become automation test coders. These testers will evaluate automation tools and select the tool that has more voting in a polling conducted by ***interviews.com forum. A framework that Google shows first would be chosen as the best suited one. A PoC that automates the login page of their application. Now stage is set to automate 90%  of test cases. NOTE: 100% is not possible!!!

This is something similar to a sip test and home use test of an edible products, there was an interesting analysis about the failure of New coke in the book Blink. I too had many snap judgment failures and one among them was selecting a gaming console. In one of my earlier jobs where I was a game tester, was given an option to join either Xbox or PS2 team on my comfort with the consoles. I tried both for about ten minutes and I felt comfortable with Xbox, so moved to Xbox team. But later when I played for more than 2 hours continuously, I realized it is not comfortable at all, later found PS2 was more comfortable for longer time duration.

Without anticipating all such problems, a set of testers who were either bored by the kind of testing they do or misguided about automation testing, would become an automation engineer without much programming skills. They would then write buggy codes to find bugs in an application that was developed by skilled programmers , read the blog title again. 😀

In Kung Fu Panda II, Po finally stops the thing that stops kung Fu after finding a secret . The secret here is to invest in developing thinking, analytical skills of testers not just on automation tool.

Automation can help to solve the problem, but that is not the solution for all problems.

PS : kung fu panda 2 – An abandoned villain threatens, there exists an unknown secret, Po sets out  a mission to understand the secret to defeat the villain and bring peace. It is twice filled with attractiveness and awesomeness …

Purpose (Dis)Solved

This is Manual Locker 🙂 . He takes 2 to 3 seconds to lock and almost the same time to unlock. Once you pull him to left you can clearly see that he has locked the door and you feel very secure. Purpose Solved

And this is hi-fi one click automation locker 🙂 this guy needs just one click to lock. But do you ever feel secure after using this lock? At least I am not. I always test this lock first by locking it with doors kept open and try to open it from outside to make sure the lock works 😀

The same is true with manual and automation testing. How many of us feel comfortable after the so called automation scripts run and say the script passed?

What other benefits does  automation lock provides than manual lock? I don’t think anything great in this context.

Both the locks solve the same purpose with no added benefits by automation. Yes, it may save 1 to 2 seconds, but considering the resources and time invested to design and manufacture the automation lock, was it worth investing? I doubt

Not convinced? Read further to know what happens if automation lock fails miserably. One sad day my friend got stuck inside bathroom for close to one hour when the critical spring inside the lock broke. So there was no option other than breaking the lock to release him :D. Purpose dissolved

Ok, now take a look at the amount of resource wasted. Also wondering how much time  wasted to design such a fancy lock. At the same time the manual lock is very simple, uses less resource, simple design and solves the desired purpose neatly.

This is how automation testing done at most places, if your manual testing solves the purpose what is the need for automation testing?

Phew! Here are the reasons to do such fancy automation

1. Attract the client by projecting automation
2. Attract testers by show-casing automation projects (testers are blindly believing it)
3. Resource utilization, if there is not much testing due to delay in development put them in automation
4. “Succumbing to The Golden Elephant Syndrome: James Bach calls one of the terrible pathologies of testing. A white elephant may be a big, useless thing, but if it’s made of gold and costs a lot of money, it’s tempting to some people to try to use it anyway. Expensive tools can cause a lot of trouble if they are badly designed and unreliable. If such a tool were cheap, we wouldn’t hesitate to throw it away. But if it’s expensive, the person who bought it doesn’t want to look like a fool-and thus becomes a bigger fool “ – this is an extract from Perfect Software:And Other Illusions about Testing

Think again!

Instead, why don’t we allow the testers to explore, question the product, so they might uncover some critical bugs, which client might discover later. Such exploration might bring in new ideas, suggestions and enhancements to the product that might help to better your competitors. Encourage testers to improve their critical thinking and analytical skill .Conduct bug battles and encourage them to write articles, blogs at least within the organization.

From my experience everywhere regression automation is a fancy way to attract clients. For instance if you goal is to reach the top floor you may take elevator instead of staircase, so you reach quicker. But at most place goal is to walk but most try to use elevator to solve the purpose.

Think Again!

I can’t conclude any automation article without referring Jonathan Kohl’s  “Test automation shouldn’t be a goal; test automation helps you achieve goals” . Read More Here

If you really want to achieve something out of automation read below articles first

http://www.kaner.com/pdfs/testarch.pdf

http://www.satisfice.com/presentations/agileauto.pdf

Note : Read this relevant and a wonderful post by Umesh Gobinath about User experience and usability here

Unbelievable! Up to 80 % off*

* -Conditions Apply

Looks like an attractive festive seasonal sale offer advertisement? Exactly I meant that. But wondering why such title here? Read further.

Most of us on seeing such attractive advertisements go to the sale with lot of planning and enthusiasm to get many items cheaper. But on entering the shop any item we pick will always have zero or very minimal off plus lot of conditions attached to it. [Huh, now you realize that the advertisement says UPTO 80% and a tiny astrix attached to it]

Similarly GUI automation tool advertisement also say it can reduce regression testing cost by up to 80% or such similar tag lines, just by showing a login page automation. I was shown with the similar login screen example in almost every session, demos or presentation I participated. Mostly it goes like this

The presenter would ask to give the login screen of our application. The he would record a script to input the login details, after a successful login he validates the home page. Now he will go and alter the script so that it validates for incorrect login details. Then he shows data parametrization options for many data and proudly says we have automated so many scenarios for a login page in no time. Unfortunately, managers who do not have automation experience were made to think that automation could be so simple and will recommend such GUI tools. Because they see something like 80% cost reduction at that point.

But the first thing I do on seeing such demo is to enable remember password option and then run the script, now the script fails because it can’t find the login page. Presenters will mostly reply that we need to apply some conditional statements to handle that (Now did you get the first condition applied?).Fine, then my application opens a Change Password page on password expiry, the script fails because the script always checks for home page after login. Presenter will think for a while and would say that we need to handle this with some conditional statements. (Now did you see the second condition applied?)Fine, then my application some time gives a full page banner about new announcements or new feature added, like how our online banking starts displaying investments plans or new service offers. Yes you readers got it correctly this is the third Condition applied. Manager also by this would have realized that the offer is not really worth 80% he might recalculated to 60%.

Next, here is another big question I ask, what if all the three of the above appears or two of them appear at any given point of time. Such cases we need to consider the order in which they appear also. Can we automate them in no time?

No, we can’t automate them just like that. We need to write functions, conditions checks and also loops, realize that this is getting into kind of programming by itself. So we now not only write the script but need to be tested as well. [We all know very well that as per D’cruz Defecto Injection law (DDIL) bug count increases exponentially for every conditional block coded. Hey wait don’t Google for DDIL because it do not exist. Then wondering why I mentioned that, because I want you guys to read Pradeep’s post Black Viper Testing]

Now calculate how many such conditions would need to be handled while automating the entire application. Manager realizes that he definitely needs to rethink on the offer that the tool will return. Now did you realize that the festival sales offer and GUI automation tools offer are the same? But wait I got one difference between them, in festival sale Up To 80% means 0 to 80% but in automation this can even go negative 🙂

A good amount of Regression Checks could be automated, but that need a solid approach, planning and good investment (both time and resource). Successful automation can’t be done as a sideline activity with testers who don’t have technical or programming skills.

Don’t Assume that Smart Is Always Smart

Smart identification is way by which QTP identifies an object when it is not found in the object repository during the run session. But it is not advisable to use this feature. Because QTP assumes and finds the object during run session, it is not a good idea to assume any think in software testing.

Let us see a simple example how QTP smart identification fails.

Scenario

I want to verify a simple web page which comes, says after I perform some transactions in a website. The web page has a simple text message “Your transaction is successful” after a successful transaction and “Your transaction is not successful” after an unsuccessful transaction. Here you are automating a scenario to check a successful transaction page exists or not after a successful transaction.

The code to check that is

If browser (“Smart is not Smart”).Page (“Smart is not Smart”).WebElement(“Your transaction is successful”).Exist Then

reporter.ReportEvent micPass,”Pass”,”Result Pass”

Else

reporter.ReportEvent micFail,”Fail”,”Result Fail”

End If

This is the sample of that page

And the result is

But now consider that the transaction is not successful and you got a message “Your transaction is not successful”

Here is the sample page

But if you run the above code, the result will be still passing.

What cause this problem? Mr. Smart Identification

The only difference between those web element in successful and unsuccessful transaction is the text. So QTP assumes both the object is same and says pass. Yes, it gives a warning on using smart identification but it is visible only in the QTP result window. If you are using some other external means to document the result, this will never get noticed.

So it is always better to disable smart identification, the simplest way to do is go to File –>Setting–>Run tab and check “Disable Smart Identification during Run Session”

If you want to disable smart identification in you Quicktest Automation Object Model, the script is

Dim App ‘As Application

Set App = CreateObject(“QuickTest.Application”)

App.Test.Settings.Run.DisableSmartIdentification = True

Here is the result after disabling the smart identification.

What Matters Most Is Focus

This is an attempt to write a sequence of articles under “What Matters Most?”

This is to share my experience, how different thinks matters in the field of software testing to bring effective and efficient solutions.

Here is first such an attempt to see how Focus matters.

Of late, I am very addictive to Farmville, a small interesting game in Facebook. Let me brief about this game. This is a simple and fun game where you do farming, add your facebook friends as neighbors, and move up in the game. You can send gifts to your neighbors from the list of available gifts. You need to spend only 5 to 10 minutes a day to make most fun out of this game. It is easy to send same gift to all the neighbors. But as my neighbor list grows, and as each of them wants a different gift it takes longer time to send them the gifts. Suddenly the automation tester within me woke up and said “automate this” (he wakes up very rarely 😉 though). I said wow! That’s a nice idea, let me attempt this and within half an hour, I got a very effective solution. The idea is

1. I need to get the entire request from friends and store it in a single file.

Solution: Oh! There is Google, ask all my friends to update the worksheet in Google Docs, with their wish list

2. Now read this file and start sending their gifts

Solution: Any automation tool can do this by reading the worksheet and schedule this run every day

Now base solution (no no I am not talking chemistry here 🙂 ) is ready, start building on this by adding all the other features like

a. What if gift asked by the neighbor is locked? (The list of gifts gets unlocked one by one as you move up in the game)

b. Send a mail back to neighbor saying that the gift you asked is locked

c. Club similar gift and send them in a single attempt (improve efficiency 🙂 )

This made me to wonder, how did I came up with such a solution so quick? This rarely happens in my projects 😉 . So I started analyzing how I got this solution?

It is “Focus” that made this possible. I just focused on one think need to automate this. I did not think too much on other factors like what tool to use? , Schedule, resources, etc.  Of course, all those are important but focus on them later, to get the solution just focus on one goal at a time (literally goal as in football, can be scored only one at a time 🙂 ). To emphasis more on this, my initial focus on just automating and not concentration on other factors naturally helped me to get other solutions as well.

So, Focus on Mission

Happy Focusing! (I should thank Pari, for adding such greetings at the end of the articles)

General Run Error in QTP

Many times  QTP throws “General Run Error” without any other details.It is always easy to debug if the system throws the appropriate error message.So how to get such error description?

QTP provides a Utility statement called DescribeResult to get the error description.

The syntax is DescribeResult(Error), argument is integer and return value is String

You can get the error number by Err.Number

So to see the description of the error,insert a watch like DescribeResult(Err.Number) in the debug pane or store it in a variable and analyze the error to fix the issue.

There are many such Utility Statement,to uncover more such utility statement go to

QuickTest Professional Help File–>QuickTest Object Model  Reference –> Utility Objects –> Utility Statements

DescribeResult

Automation Search Engine

A New automation search engine from automated testing institution customized by Google. www.googleautomation.com

For more info look at http://tinyurl.com/m7ad8f

Count Number of links in a Web Page

In this let us see a script to count number of links in a web page.This will help to understand the descriptive programing better and will help to solve many practical requirements like finding the broken links.

Let us count the number of links in google’s home page

Set mygooglesearch = Description.Create()

Then you declare the property name and value pair for that object,here since we are counting the link,we have to use micclass property  of type link.

mygooglesearch(”micclass”).value =”Link”

Now use childobjects to get the properties into link_count objects,this will give all the child objects of that page of type link.

Set link_count = Browser(“name:=google”).Page(“title:=google”).childobjects(mygooglesearch)

Now we get the object count using count method.

msgbox link_count.count

Simple Parametrization in QTP

Parameterization is used when we need to handle run time data in scripts, consider a scenario of verification of user login to gmail. We verify the user name displayed at the top right corner to make sure that the user logged is correct. But the name displayed depends on the login id used and is not static, so we need to parameterize the value.

Parameterization may be done by passing values as variable or through data sheet. It is always a good practice to user data sheet, so that editing the data during play back and maintenance will be easier.

Below is the sample code for parameterization through variables

Id = “<emailid@gmail.com>”

Pwd= “<password>”

Browser(“Google”).Page(“Google”).Link(“Sign in”).Click

Browser(“Google”).Page(“Google Accounts”).WebEdit(“Email”).Set(Id)

Browser(“Google”).Page(“Google Accounts”).WebEdit(“Passwd”).SetSecure(Pwd)

Browser(“Google”).Page(“Google Accounts”).WebButton(“Sign in”).Click

‘ Verification point verifies whether used has successfully logged in by checking the email displayed after logging in

Login = Browser(“Google”).Page(“iGoogle”).WebElement(“innertext:=”&Id).GetTOProperty(“innertext”)

msgbox(Login)

If Login=Id Then

Browser(“Google”).Page(“Google”).Link(“iGoogle”).Click

Browser(“Google”).Page(“iGoogle”).Link(“innertext:=Sign Out”).Click

Else

msgbox(“Login Failed”)

End If

Below is the sample code for parameterization through datasheet

Browser(“Google”).Page(“Google”).Link(“Sign in”).Click

Browser(“Google”).Page(“Google Accounts”).WebEdit(“Email”).Set(DataTable(“EmailID”, dtGlobalSheet))

Browser(“Google”).Page(“Google Accounts”).WebEdit(“Passwd”).SetSecure(DataTable(“Password”, dtGlobalSheet))

Browser(“Google”).Page(“Google Accounts”).WebButton(“Sign in”).Click

‘ Verification point verifies whether used has successfully logged in by checking the email displayed after logging in

Login = Browser(“Google”).Page(“iGoogle”).WebElement(“innertext:=”&DataTable(“EmailID “, dtGlobalSheet)).GetROProperty(“innertext”)

msgbox(DataTable(“EmailID”, dtGlobalSheet))

If Login=DataTable(“EmailID”, dtGlobalSheet) Then

Browser(“Google”).Page(“Google”).Link(“iGoogle”).Click

Browser(“Google”).Page(“iGoogle”).Link(“innertext:=Sign Out”).Click

Else

Message = Browser(“Google”).Page(“Google Accounts”).Webelement(“html id:=errormsg_0_Passwd”).GetROProperty(“innertext”)

msgbox(Message)

If Message = “Username and password do not match” Then

msgbox(“Login Failed”)

Else

msgbox(“Invalid Error message”)

End If

End If

Parameterization is used when we need to handle run time data in scripts, consider a scenario of verification of user login to gmail. We verify the user name displayed at the top right corner to make sure that the user logged is correct. But the name displayed depends on the login id used and is not static, so we need to parameterize the value.

Parameterization may be done by passing values as variable or through data sheet. It is always a good practice to user data sheet, so that editing the data during play back and maintenance will be easier.

Below is the sample code for parameterization through variables

Id = “<emailid@gmail.com>”

Pwd= “<password>”

Browser(“Google”).Page(“Google”).Link(“Sign in”).Click

Browser(“Google”).Page(“Google Accounts”).WebEdit(“Email”).Set(Id)

Browser(“Google”).Page(“Google Accounts”).WebEdit(“Passwd”).SetSecure(Pwd)

Browser(“Google”).Page(“Google Accounts”).WebButton(“Sign in”).Click

‘ Verification point verifies whether used has successfully logged in by checking the email displayed after logging in

Login = Browser(“Google”).Page(“iGoogle”).WebElement(“innertext:=”&Id).GetTOProperty(“innertext”)

msgbox(Login)

If Login=Id Then

Browser(“Google”).Page(“Google”).Link(“iGoogle”).Click

Browser(“Google”).Page(“iGoogle”).Link(“innertext:=Sign Out”).Click

Else

msgbox(“Login Failed”)

End If

Below is the sample code for parameterization through datasheet

Browser(“Google”).Page(“Google”).Link(“Sign in”).Click

Browser(“Google”).Page(“Google Accounts”).WebEdit(“Email”).Set(DataTable(“EmailID”, dtGlobalSheet))

Browser(“Google”).Page(“Google Accounts”).WebEdit(“Passwd”).SetSecure(DataTable(“Password”, dtGlobalSheet))

Browser(“Google”).Page(“Google Accounts”).WebButton(“Sign in”).Click

‘ Verification point verifies whether used has successfully logged in by checking the email displayed after logging in

Login = Browser(“Google”).Page(“iGoogle”).WebElement(“innertext:=”&DataTable(“EmailID “, dtGlobalSheet)).GetROProperty(“innertext”)

msgbox(DataTable(“EmailID”, dtGlobalSheet))

If Login=DataTable(“EmailID”, dtGlobalSheet) Then

Browser(“Google”).Page(“Google”).Link(“iGoogle”).Click

Browser(“Google”).Page(“iGoogle”).Link(“innertext:=Sign Out”).Click

Else

Message = Browser(“Google”).Page(“Google Accounts”).Webelement(“html id:=errormsg_0_Passwd”).GetROProperty(“innertext”)

msgbox(Message)

If Message = “Username and password do not match” Then

msgbox(“Login Failed”)

Else

msgbox(“Invalid Error message”)

End If

End If

What is Regular Expression?

What is Regular Expression?

Regular expression is a set of symbols (characters) and syntactic elements used to match patterns of text. The term regular expression is often abbreviated to regex or RE.

A regular expression is a way of describing a string of text using metacharacters or wildcard symbols.

A regular expression, often called a pattern, is an expression that describes a set of strings. They are usually used to give a concise description of a set, without having to list all elements.

Quantification:

A quantifier after a token (such as a character) or group specifies how often that preceding element is allowed to occur. Given below are most commonly used quantifiers

1. Period (.) – Matches one character in its position any character(including numbers,special    characters) other than /n(new line)
Example : welcome.qtp matches “welcome@qtp”,”welcome2qtp”,”welcome qtp” but not “welcomeqtp”
2. QuestionMark(?) – Matches zero or one occurances of previous character.
Example: goo?gle matches “gogle” and “google”
3. Asterisks(*) – Matches zero,one or many occurances of previous character.
Example: goo*gle matches “gogle”, “google”,”gooogle”,”gooooooooooooooooooogle” etc..
4. Plus(+) – Matches one or more occurances of previous character.
Example: goo+gle matches “google”,”gooogle” etc.. but not gogle
5. Backslash (\): It can be used in conjunction with a special character to indicate that the next character be treated as a literal character
Example: welcomenew\* will literally match “welcomenew*”
6. Caret (^) – Match any character in the list except for the ones specified in the string
Example: [^c]at matches other than cat like bat,rat,@at,2at etc..
7. Pipe (|) – Logical OR
Example: (c|r)at matches cat or rat
8. Square Braces([]) – Matches the range for anything under that range
Example : [a-z] all lower case,[A-Z] all upper case,[0-9] numbers for 0 to 9

Let us try to come up with a regular expression which allows numbers from 0 to 1200 (this is given in QTP help).We can solve this by having three different patterns first for 0-999,1000-1199 and 1200

( [0-9][0-9][0-9] | [1][01][0-9][0-9] | 1200),but the problem here is it will allow to only enter four digits ,so 7 should be entered as 0007 similarly 77 as 0077,so to solve that use ? Quantifier ([0-9]?[0-9]?[0-9] | [1][01][0-9][0-9] | 1200).

The Regex Coach is a graphical application for Windows which can be used to experiment with regular expressions interactively. You can download this from http://weitz.de/regex-coach/