I love talking people from the software industry specially those that have crazy definition on the automation in QA process. Let me just outline some of the recent ones that I underwent.
Hey Hi Vip how are you doing as an automation tester ?
I am an automation tester primarily responsible for automating test cases related to UI and also database test automation.
Oh great so what is it that you can automate ?
I can automate set of test cases that are written as part of test cases within a test suite.
Oh that sounds really good , so how many test cases can you automate in a day ?
Well, the answer is very simple . I cannot give you an exact or even a near count of the same.
Why, but you said you are an automation tester and primarily responsible for automating test cases related to UI.
So now tell me from your experience how much is it that your productivity is in terms of automating the test cases ?
It is really not a feasible thing to answer unless I see the test suite that needs to be automated.
Do you mean to say that the estimation can be given on the basis of the test suite and the test cases it contains ?
Of course that is what I mean to say, because when you have a test suite , it is then that an automation tester can give you an estimate , because the automation tester does not believe in stories in terms of estimation.
There are several parameters that need to be assessed before it can be concluded as to what sort of breakdown in terms of the test script point needs to be done to provide an estimate that shall hold relevance and instill confidence in other stakeholders.
I thought only project managers give such long stories when it is the estimation time for project schedule, but now even automation testers have grown in same shoes.
Yeah just because even automation testing has grown in same lines and now you have lot of test managers available in the industry that need to take care of these things.
OK. So you were talking of several parameters on which you can provide the test automation estimation.
Oh yes at least now I could figure out that you are listening seriously to what I kept talking for last couple of minutes .
Nope , I am absolutely perfect with what you said. It is not the first time I see someone talking logical for the test automation topic, but with bandwidth available, we can take it to some extent. So what are those parameters.
Yeah, basically no automation suite targets execution of every test cases, it is just a subset of the test cases that is taken into account when we target automation.
Now the criterion of that subset creation can be decide by three set of people.
Oh that seems to be going interesting now - Subset of test cases and then subset of people.
Yes, so one from technical category on feasibility from technology front as to which tool can be used in current application in terms of support, for example if we target automation of a UI that has SAP stuffs and trying to use the QTP , we need the SAP add in with QTP licenses. This person shall also keep knowledge on pricing of tool in terms of script development timelines , the coding complexity and the resources available ready made or if not then time needed to work on ramping up people by getting them trained by professionals or self coaching. Another parameter here can be in terms of coding language efficiency of the script writers for example if you want to use the IBM rational tool, you should have people who are well versed with JAVA.
Now that sounds a bit wary, would you also say that the testers who will do automation scripting should have programming skills.
Oh yes that is an entry criterion to build up the team. Strong logical thinking is a must or else you would end up no where. But then a lot has been done in this area by making tools more friendly with users , such as the one that Visual studio offers with its Coded UI test classes. Most of the things do get auto generated , and if you have 7/10 rating in programming language , you can learn in short timelines. But analytical and logical reasoning skills are something that should be in abundance.
Why do you think so? I think with the test suite available you do not need to have those skills as the automation tester need to just script down what is manually written in test cases ! Or am I missing something here ?
Yes dear you are missing something greatly. It will impact the execution timelines hugely if you donot organize the test cases while writing test scripts as you would not like to come to same page again and again if you have that analytical ability to create scripts in a manner that the number of times you do same set of activity gets minimized.
I still cannot understand this parameter dear. People and technical parameter is well understood though.
Ok. So a basic thing I ask you. If I ask you to go to shop and buy some medicine for the mild headache now I am having due to your dull brain, what would you do? I know being a good friend of mine you would do the needful. But then I ask you again to bring some sandwitches from the shop which was on the same way. Do not you feel irritated ?
Oh yes I would blast at you. Why the heck you did not tell me when you ask me to bring medicine, as it is on same street ?
Exactly.Yeah now you are getting it. So plan it upfront which scripts need to be run in continuation with another script rather doing hell lot of round trip. You got it this time !
WOW.
Hmm, my dear friend now understand when he himself is forced for un-necessary round trips. So do avoid these by bringing in your strong analytical and logical skills during the course of script development and integration of several modules. This does come up handy when you have certain things that get created in Module A and consumed in some other module B or at times in multiple modules.
There are indeed lot of things that need to be taken care of before estimating. It is not that simple though it seemed from the outset .
Good now you will listen to what I say.
So what else do you guys take into account ?
Now it is the simple breakdown of the test cases that has been identified as a part of Automation test suite.
Breakdown of test cases ? What is it that you will do now ?
One very simple thing we all understand is here :
All test cases will not have same level of technical complexity involved while scripting.
All test cases will not have same number of steps in it.
So some sort of normalization will come in picture.
Technical complexity is something that may not be estimated in absolute precision , but with experience this can be accounted for and kept as separate addition to the estimated timeines.
However the number of steps can be estimated in clear calculation.
For example :
Simple test cases - Upto 7 steps
Medium test cases - 8 -15
Complex test cases - 16 -23
Above 23 ? What will you do then ?
Oh you are listening logically and analytically using your brain. Cool . Generally I do not prefer to have such test cases as one test case, but if at all we are in a scenario where we are not the author of the test cases, it will be done as under
30 test steps = 1 complex test case + 1 simple test case. And so on and so forth.
Now that sounds a bit wary, would you also say that the testers who will do automation scripting should have programming skills.
Oh yes that is an entry criterion to build up the team. Strong logical thinking is a must or else you would end up no where. But then a lot has been done in this area by making tools more friendly with users , such as the one that Visual studio offers with its Coded UI test classes. Most of the things do get auto generated , and if you have 7/10 rating in programming language , you can learn in short timelines. But analytical and logical reasoning skills are something that should be in abundance.
Why do you think so? I think with the test suite available you do not need to have those skills as the automation tester need to just script down what is manually written in test cases ! Or am I missing something here ?
Yes dear you are missing something greatly. It will impact the execution timelines hugely if you donot organize the test cases while writing test scripts as you would not like to come to same page again and again if you have that analytical ability to create scripts in a manner that the number of times you do same set of activity gets minimized.
I still cannot understand this parameter dear. People and technical parameter is well understood though.
Ok. So a basic thing I ask you. If I ask you to go to shop and buy some medicine for the mild headache now I am having due to your dull brain, what would you do? I know being a good friend of mine you would do the needful. But then I ask you again to bring some sandwitches from the shop which was on the same way. Do not you feel irritated ?
Oh yes I would blast at you. Why the heck you did not tell me when you ask me to bring medicine, as it is on same street ?
Exactly.Yeah now you are getting it. So plan it upfront which scripts need to be run in continuation with another script rather doing hell lot of round trip. You got it this time !
WOW.
Hmm, my dear friend now understand when he himself is forced for un-necessary round trips. So do avoid these by bringing in your strong analytical and logical skills during the course of script development and integration of several modules. This does come up handy when you have certain things that get created in Module A and consumed in some other module B or at times in multiple modules.
There are indeed lot of things that need to be taken care of before estimating. It is not that simple though it seemed from the outset .
Good now you will listen to what I say.
So what else do you guys take into account ?
Now it is the simple breakdown of the test cases that has been identified as a part of Automation test suite.
Breakdown of test cases ? What is it that you will do now ?
One very simple thing we all understand is here :
All test cases will not have same level of technical complexity involved while scripting.
All test cases will not have same number of steps in it.
So some sort of normalization will come in picture.
Technical complexity is something that may not be estimated in absolute precision , but with experience this can be accounted for and kept as separate addition to the estimated timeines.
However the number of steps can be estimated in clear calculation.
For example :
Simple test cases - Upto 7 steps
Medium test cases - 8 -15
Complex test cases - 16 -23
Above 23 ? What will you do then ?
Oh you are listening logically and analytically using your brain. Cool . Generally I do not prefer to have such test cases as one test case, but if at all we are in a scenario where we are not the author of the test cases, it will be done as under
30 test steps = 1 complex test case + 1 simple test case. And so on and so forth.
Suppose my automation suite has 50 test cases
Simple - 15
Medium - 25
Complex - 10
I will do some normalization over here to calculate some test case point in simple terms by associating some weight factors as decided by the team.
Weight factor for Simple - 1
Weight factor for Medium - 3
Weight factor for Complex - 5
Now my suite in terms of Test case point will look like this :
15*1 + 25*3 + 10*5 = 140
On an average it is 30 Test script point for 1 Man day.
So the scripting timeline would be 140/30 = 5 Man days + some buffer for technical complexity
Are you awake buddy ! Good keep sleeping ...
So that is how it is and these numbers are not absolute , but they are relative, I just gave you an overview on how we do estimate when some automation testing need to be done. In ideal cases we should target a POC first and check for all feedback in terms of code modularization, multiple environment in which the execution need to be done - such as run of the test scripts in two URLs simultaneously, additional features such as cross browser script run, logs for the executed test scripts with time stamp. These feedback should be addressed within the POC phase so that no surprise springs up during the course of scripting.
إرسال تعليق