Think different but keep it simple :)

There is content below, I can read can’t you? oh!  Think different to see the content, but keep it simple 🙂

This is the most common problem we testers encounter quite often, it works in developer machine but not in test machine.

Once an interesting incident happened in one of the web application I was testing, application was working fine in developer’s environment but not in test environment.

The most likely reasons could be either  the test servers do not have the latest code or  there is conflict in environment. Quick check revealed that both the environments had the same code.

Now the focus was on to investigate the configuration of the servers.

As a tester how will  you investigate such situations?

Few tips to nail down such problems. Please share your tips as well.

1. Test in another test machine\server

2. Test in another dev machine\server

This will help to narrow down if there the problem is at client side or server side.

3. Check if  that machine  is also hosting the web server.

Time out issues and session management related issues may not be reproducible, if the machine you are testing in is a server machine.

4. Check for additional softwares installed.

Generally developer box will have lot of additional softwares installed, like debug tools, development related softwares that may prevent from the bug to be reproduced. Even the web browser in dev box will be loaded with tons of additional tools and utilities. Such additional installation may also prevent from some bugs being reproduced.

It is not enough to check if required softwares were  installed, it is equally important to make sure no additional unwanted softwares were installed.

Coming back to the issue, following those above mentioned approaches by pairing with developer helped to nail down the problem in few hours. The issue was, a bulk data extraction component used (it was ETL project) will work only in localhost, that does not have the capability to connect with external host. In the developer machine and dev server both web server and DB server was hosted  in the same machine, so that worked perfecly fine.But test environment had DB and Web server running in different machine, so this caused the problem. An important lesson learned from the incident is it is always good to have the test environment mimic the production. Keeping the test environment similar to production helped in finding the problem very earlier in the cycle.

Another good practice to follow, at any given time at least one test environment should be  in sync with production server, so any production issues reported can be easily replicated and analysed in a test environment. Having only one test environment means most time that will be updated with enhancements going on, that may also prevent from reproducing production issues because of code changes applied in the test environment.

How many of you have set up the test environments? No, I didn’t mean double clicking an installer provided by the developer. As a tester I learned a lot, by setting up test environment and maintaining it, setting up test environments will help  to understand your application better, it will help to get better test ideas, analyzing of bugs can be done  effectively .

Understanding test environment improves testing.

——————————————————————End of Post—————————————————————————–

iTest

Last week I participated in a competition conducted by 99tests.com, a crowd sourced testing start-up from India.  This is  my approach\experience report on how I tested and won the competition.

What were the givens?

No requirement, no design documents, no test cases, no use cases, no user stories. All I had were credentials to login and the URL to test. So, do I need to agitate and not to start testing until I get requirements? Do I sit and write test cases? Any conventional scripted testers would’ve struck wondering what to do. So, here is a warning for all those scripted testers it looks things are changing because of agile approach and such crowd sourced testing services.

How did iStart?

I didn’t waste any time exploring the application. I thought the best approach with the givens (actually no givens ;-)) would be, jump straight into using the application and observing the behavior. I thought that would help with forming test ideas and then to build from there.

Being a user of such online shopping sites helped me to frame some initial expectation. So, started with Follow the (user) Flow heuristics. I decided to create my own credentials instead of using test credentials and login. Then observe how easy or buggy it is to find an item, add to cart, check out, paying through a third party payment gateway and choose a shipping address of my choice.

Did iFollow the flow?

Registration was successful  upon giving correct details. But I didn’t get to see any error message, so decided to enter some invalid data to find the application behavior. What I found in giving invalid input was the error messages were not user-friendly, not just in terms of the message, in terms of usability as well. Error messages were in a different page and users needing to click back button to get back to the registration page. On click on back button the entire data entered were lost. So, this made me take a deviation from my initial plans of Follow the flow. Decided to test this module thoroughly, wondering why? Read bug advocacy of this bug below.

How iDid bug advocacy?

I didn’t just log the bug with a summary line, description, steps to reproduce and screen shots. Instead, I also explained how and why this might bug the users and impact of the bug.

The user registration page plays a vital role in giving the first impression about the application behavior. Also, users use online shopping sites mainly to save time, by displaying errors in another page and asking the user to come back to previous screen to correct the data is actually wasting sufficient time. This advocacy helps in understanding the real impact of such bugs. So, always explain how any bugs\issues found would potentially affect the user experience.

Your bugs are your representatives. The bug logging also depends on context, if you know your developers very well and if he is sitting next to you, the bug logging, or advocacy may differ from the way you log bugs in crowd sourced testing. Here you have no clue about, who is doing bug triage, developers and their understanding of the product. So always give as much detail as possible so that they can’t reject your bugs.

Always remember this  most famous movie dialogs of all time, from “The God Father” movie while logging the bugs

“I’m going to make him an offer (details) he can’t refuse (to fix).”

Did iJust log bugs?

No, I went through most of the bugs logged by other testers. I posted comments, raised questions where ever I felt the bug was really not a bug or if the priority was inflated or if the issues were duplicated. Also, I neither missed to appreciate some good bugs reported by other testers nor missed to learn from fellow testers. I was actually a little disappointed that there were a significant amount of bugs without clear description and duplicates.

What were my objectives?

Than winning the competition, my objective was to log the maximum number of valid bugs. Ended logging 50 valid bugs, maximum by any tester. Also wanted to maintain a high bug acceptance ratio, 86% of the bugs were valid. Happy with that but still wanted to improve on acceptance ratio.

So, What iDidn’t test?

I found few even tested the Facebook ‘Like’ gadgets, Payment gateway and spell checks in the application’s blog. Though few of such bugs were valid, those were not from the application under test, so they got rejected. So,

Know your boundaries, so you do good enough testing in given time.

Ready to try some bug advocacy?

Here are a few other bugs logged by me, try to advocate for them. The application under test was an online shopping site.

  • “Similar Items” feature is missing.
  • Same book title, but displayed with huge difference in price tag.
  • The amount should always be right justified.
  • On entering the special characters the system through an “invalid gift message”
  • Pre order items are shown as Available and Buy Now.
  • Is the final price displayed in product description inclusive of taxes?
  • Can’t store search results.

Happy Bug Hunting!

P.S.

Some of my favorite lessons in Bug Advocacy chapter of “Lessons Learned in Software Testing”  book

55: You are what you write.

58: Your bug report is your representative.

83: The summary line is the most important line in the bug report.

89: Use market or support data when appropriate.

101: When you decide to fight, decide to win!

Simple application, Simple Bug(?) and a Simple Investigation

NOTE : If you ask how simple is it? it depends 😉

Twitter changed the way I communicate. It is simple, sweet, short and faster way to share information.

Recently I started using HootSuite, a social media tool to manage my social networking activities. This has a nice little option to view twitter profiles. On clicking a twitter handle opens a pop up that displays bio, picture options to reply, DM, follow, unfollow and few others. I clicked on my own twitter profile. To my surprise that also has follow, unfollow, DM options enabled. So first tried to Follow myself (a tester in full swing 😉 ) got a message “You can’t follow\unfollow yourself” hmm that looks good.

Then, I clicked on Direct Message(DM) to myself and was able to send a DM successfully to myself. I am not sure if this is the correct behavior or is a problem? If it is a problem, where is the problem?

I never tried sending DM to myself from twitter, so next tried to send DM to myself from twitter.

From the Direct Message options, tried sending a DM, it was not allowed.

Next tried with other option available to send DM, home page directly.

Oh! it is possible and the DM delivered to my twitter inbox. So there is inconsistency in handling DM to oneself in twitter. So, it is a problem.

So,as  a tester should look for as many details as possible like trying out all possible input and output options at API level, DB levels and UI levels, before logging a bug. That will help report the problem clear thus issue might get fixed faster and better.

Above all, first thing you should do on finding an issue is check if that is already logged. If yes, compare with your observation with those details, and provide information if you feel something is missed out.

Exploring more on this, Twitter help on DM says DM cannot be send to a user who is not following you and there is no mention of DM to yourself.  Since I can send a DM to myself but I am not following myself, so does this mean the claim made by twitter that “you cannot send a DM to a user who is not following you” is wrong?

What is your view?

Now your turn to investigate the not so user friendly message.

Below error message was from Windows XP with SP2, this happened while renaming a normal windows  folder. As a user what did you understand from this message? And what actions would you perform as a user to solve this problem?

Here is the investigation report of above Windows bug

Click here to see more such bug(?) posts.

Share

Don’t Assume that Smart Is Always Smart

Smart identification is way by which QTP identifies an object when it is not found in the object repository during the run session. But it is not advisable to use this feature. Because QTP assumes and finds the object during run session, it is not a good idea to assume any think in software testing.

Let us see a simple example how QTP smart identification fails.

Scenario

I want to verify a simple web page which comes, says after I perform some transactions in a website. The web page has a simple text message “Your transaction is successful” after a successful transaction and “Your transaction is not successful” after an unsuccessful transaction. Here you are automating a scenario to check a successful transaction page exists or not after a successful transaction.

The code to check that is

If browser (“Smart is not Smart”).Page (“Smart is not Smart”).WebElement(“Your transaction is successful”).Exist Then

reporter.ReportEvent micPass,”Pass”,”Result Pass”

Else

reporter.ReportEvent micFail,”Fail”,”Result Fail”

End If

This is the sample of that page

sucessful

And the result is

pass

But now consider that the transaction is not successful and you got a message “Your transaction is not successful”

Here is the sample page

unsuccessful

But if you run the above code, the result will be still passing.

pass-warning

What cause this problem? Mr. Smart Identification

The only difference between those web element in successful and unsuccessful transaction is the text. So QTP assumes both the object is same and says pass. Yes, it gives a warning on using smart identification but it is visible only in the QTP result window. If you are using some other external means to document the result, this will never get noticed.

So it is always better to disable smart identification, the simplest way to do is go to File –>Setting–>Run tab and check “Disable Smart Identification during Run Session”

If you want to disable smart identification in you Quicktest Automation Object Model, the script is

Dim App ‘As Application

Set App = CreateObject(“QuickTest.Application”)

App.Test.Settings.Run.DisableSmartIdentification = True

Here is the result after disabling the smart identification.

fail

General Run Error in QTP

Many times  QTP throws “General Run Error” without any other details.It is always easy to debug if the system throws the appropriate error message.So how to get such error description?

QTP provides a Utility statement called DescribeResult to get the error description.

The syntax is DescribeResult(Error), argument is integer and return value is String

You can get the error number by Err.Number

So to see the description of the error,insert a watch like DescribeResult(Err.Number) in the debug pane or store it in a variable and analyze the error to fix the issue.

There are many such Utility Statement,to uncover more such utility statement go to

QuickTest Professional Help File–>QuickTest Object Model  Reference –> Utility Objects –> Utility Statements

DescribeResult

QTP Tips.txt

There is a Tips.txt present in the QTP installed directory.This has a lot of useful tips.It is present under QTP Installed Directory\dat\Tips.txt.Mostly C:\Program Files\HP\QuickTest Professional\dat

Wondering how did I miss this for so long,there are many interesting and lots of unknown tips(many might be aware of it,at least unknown to me). After uncovering this file, I googled it and found one blog referring to this file. Here is that link, he has given all the tips in the blog.