One fine Monday morning, I walked across Mark who is a testers and a domain expert in the application to which I am pretty new.
ME: “Hey, café?”
Mark: “No dude just received this list of loans which I need to feed into the test system ”
ME: “Oh is it? What you do actually?”
Mark: “Here is the list of 100 odd loans that I have to feed into this using this XYZ application, so that ABC department will test this out in the build they get tomorrow.”
(Mark is the domain expert in this XYZ application, many other application talk to his application to get data for reporting, rule engines to name a few)
ME: “Hey that should be boring, is a onetime work?”
Mark: “No ME, I’ve to do this whenever any other department need some specific loan data. I have to do this almost every week for one or the other department during the release cycle.”
ME: “oops! So what exactly other departments give to you?”
Mark shows an excel sheet that has just list of loan numbers in a single column
Mark: “They just email me the list of loans they need, I just use the loan number to this main DB machine and gives the source DB so that the loan from this gets copied to the test DB.”
ME: “Is it? Can you give me a demo for it?”
On seeing the demo I understood that he navigates 4 mainframe screens then use the loan number provided and the test DB server provided by the other department to initialize jobs. He repeats this for the entire list of loans, each takes 3 minutes to enter.
ME: “What if the loan is already present in the source DB?”
Mark: “You get a message about the existence”
The first think that flashed is better to automate this. We already had the most costly automation tool with all add in for the mainframe interface. Wrote a very basic script to read the loan number and designation DB name from QTPs Data table and automate the other steps. Now each loan takes less than 30 seconds, asked him to maintain a sheet which calculates how many hours was by this script. The effort to create this script was half a day. We faced few issues in new release that were fixed using regular expression.
Next Monday morning
ME thru messenger: “Mark, café?”
Mark: “I am working from home, have your café alone”
Another not so fine someday morning, meeting with Mr.X.
Mr.X: “We need to come up with automation scripts for the entire regression test suite and deliver to the client”
ME: “Automation suite to client? But why do they need?”
Mr.X: “We have made agreement with client that we provide with complete regression automation suite so that they can run complete regression suite through automation”
ME: “But I think from the client perspective they need to test manually so they get the feel of the application, also from our experience we know very well that the automation scripts need to be debugged very often by experienced automation engineer for many problems starting from Data integrity issues to understanding the results to many such problems”
As usual without listening, my Mr.X continued
Mr.X: “We need to build a robust framework and so that the client will be able to use our framework easily”
ME: “So are we hiring another QA team to test this project, this looks like you are building software to test this software”
Mr.X: “Not exactly, but we need to look into next step in automation”
ME: “That is perfectly fine but at what cost? Whose benefit? ”
I am seeing this trend happening in many places. I really don’t understand the purpose of building a GUI automation regression scripts to be run by the client. First the trend was Regression Automation now it has another version GUI Regression Automation Suite for Clients.
But the problems here are
- As Client I would love to see\test the system that was build, so that I can make a judgment whether the system is behaving as expected. I do not how automation adds value to client.
- And no one is considering the maintenance cost. Automation even on play back needs experienced person to monitor and debug it regularly
- And the time automation tool takes to yield the returns. In Most cases it will take three or four years to yield the returns. Refer this link to understand why http://www.testingeducation.org/BBST/BBSTGUIRegressionAutomation.html
I gave those two of those incidents to emphasize that
- Automation isn’t just automated regression testing. Automation and regression involve different considerations as said by Dr.Cem Kaner.
- Look around and talk with other people to know the kind of work they do, so you may add some value to it. Also you might get some interesting ideas from them to implement in your work.
Happy Looking Around 🙂