Manual testing has always been the mainstay of any STLC. It starts with the initial stages of the Software development life-cycle. Have a look at any of the SDLC, being it the latest introduced Agile or one of the most orthodox of all the waterfall model, the STLC starts from the very begining. specific reason for this very trend in existence lies in the fact that finding a bug early in the SDLC reduces the cost to fix the same.
That definitely is not the topic of discussion in this post but I am sure we will get through those lines very often when the very terms of test execution comes to our mind.
So starting with the points I have to discuss and highlight in this post our Test execution cycle normally starts with a buddy build that gets released as a pre-test environment to help the test team adjusted with the way the functionality is going to be implemented. Does that mean as a test resource I am not going to log bugs for this very build that is available for test. That sounds so interesting a riddle that can be solved only through the experience and not just the orthodox quality assurance technique. So basically a buddy build is released to a test team not for logging bugs but to give the look and feel , and also there remains one hidden advantage of this very phase. It is meant to help the test team figure out the areas that need a change request in place from the client or product owner side.In case the product needs some change request to be implemented within the functionality and the same can be feasible as per the development team's understanding, a confirmation is sought for from the client and the same gets implemented as per the decided stack rank of the change request created. that is the whole idea of a buddy build in place. So a question that comes to the mind here is do we really bother to implement things that is not part of business requirement. And do we really have any extra mileage getting carved out by doing this extra bit of test effort that goes no way to optimize life-cycle timelines from the test execution front. Now that sounds bit harsh. But yeah there is a tweak of activities that get triggered by getting this activity implemented and scoped in the life-cycle.The very first thing that the test team gets attributed with is the bandwidth enhancement in terms of understanding the way the features and functionality is getting implemented without being accountable for not being able to find bugs prevalent in the system under test.The second is the early detection of the product shortfalls which if not detected earlier in the life-cycle might have huge impact in future releases.Third point is customer gets huge confidence in the test team's ability to understand the functionality and capability to deliver quality product not from functional aspect alone but also from the domain aspect simply because on most of the occasions customers are not very adept in documenting the requirements and huge number of change requests keep coming in future phases of releases done.
However lets move onto the topic we were to discuss upon. I have the very bad habit to create huge background for the main story to be put down into the blog.
So lets start with the test execution plan that we have in place. Suppose we have three test execution cycles , one regression cycle and finally one sanity cycle. So these are the three main phases within which the complete execution needs to be performed so to add quality and release a defect free product into the system.
For our case let us consider we have a matrix of information as per the documented activity. We have test case counts as under in terms of priority with which the test cases need to be executed:
Priority 1 - 350
Priority 2 - 650
Priority 3 - 200
That does look an ideal count for a small to mid sized project which comes under test execution cycles of three rounds.Here we assume that all the 1200 odd cases are part of the test execution in the regression cycle that needs to be performed after the three rounds of iteration that shall be performed . However, as in a typical scenario of test execution plan , for the first round of testing that needs to be carried out in the System Test cycle 1 - we will not have all 1200 odd test cases as part of execution cycle.Let us assume that for the first round we have close to 400 odd test cases as part of execution scope and their matrix is as under.
Priority 1 - 150
Priority 2 - 200
Priority 3 - 050
With this execution plan in place the System Test cycle 1 gets executed. Now let us move into the next phase namely the System Test cycle 2. Suppose here in we have close to 550 odd test cases as part of execution scope and their matrix is as under.
Priority 1 - 150
Priority 2 - 300
Priority 3 - 100
That makes test execution interesting or rather more complex than what many others feel about it !!! Do you now agree that testing a product is far more difficult than developing it.
So you end up testing all the impacted areas as part of this test execution cycle.
I have seen many projects where in the test lead does not break his/her head much into defining the impacted areas and maintaining traceability matrix , but they just end up executing all the test cases from execution cycle 1 also in the execution cycle 2.That gives clear indication of DULL work methodology and lack of SMART work , and making the job of complete test team monotonous.
Something that comes as a stumbling block typically in a Service industry based project is insufficient clarity on the documented business requirements. So in cases such as these I have my clarity in terms of Test cases and I maintain a vertical traceability on the test cases, so I get clear vision on which test case when failed and fixed might impact which other test cases.I hope that sounds effective solution model to the problem we geared up for in the current context.
Now you might have understood how the test execution becomes more challenging as the product matures more in subsequent cycles. Let us move into the next phase namely the System Test cycle 3.So this is the last phase of major development cycle and with this we will come to and end to the functionality development. Here we are left with close to 200 odd test cases as part of execution scope and their matrix is as under.
Priority 1 - 050
Priority 2 - 100
Priority 3 - 050
That definitely is not the topic of discussion in this post but I am sure we will get through those lines very often when the very terms of test execution comes to our mind.
So starting with the points I have to discuss and highlight in this post our Test execution cycle normally starts with a buddy build that gets released as a pre-test environment to help the test team adjusted with the way the functionality is going to be implemented. Does that mean as a test resource I am not going to log bugs for this very build that is available for test. That sounds so interesting a riddle that can be solved only through the experience and not just the orthodox quality assurance technique. So basically a buddy build is released to a test team not for logging bugs but to give the look and feel , and also there remains one hidden advantage of this very phase. It is meant to help the test team figure out the areas that need a change request in place from the client or product owner side.In case the product needs some change request to be implemented within the functionality and the same can be feasible as per the development team's understanding, a confirmation is sought for from the client and the same gets implemented as per the decided stack rank of the change request created. that is the whole idea of a buddy build in place. So a question that comes to the mind here is do we really bother to implement things that is not part of business requirement. And do we really have any extra mileage getting carved out by doing this extra bit of test effort that goes no way to optimize life-cycle timelines from the test execution front. Now that sounds bit harsh. But yeah there is a tweak of activities that get triggered by getting this activity implemented and scoped in the life-cycle.The very first thing that the test team gets attributed with is the bandwidth enhancement in terms of understanding the way the features and functionality is getting implemented without being accountable for not being able to find bugs prevalent in the system under test.The second is the early detection of the product shortfalls which if not detected earlier in the life-cycle might have huge impact in future releases.Third point is customer gets huge confidence in the test team's ability to understand the functionality and capability to deliver quality product not from functional aspect alone but also from the domain aspect simply because on most of the occasions customers are not very adept in documenting the requirements and huge number of change requests keep coming in future phases of releases done.
However lets move onto the topic we were to discuss upon. I have the very bad habit to create huge background for the main story to be put down into the blog.
So lets start with the test execution plan that we have in place. Suppose we have three test execution cycles , one regression cycle and finally one sanity cycle. So these are the three main phases within which the complete execution needs to be performed so to add quality and release a defect free product into the system.
For our case let us consider we have a matrix of information as per the documented activity. We have test case counts as under in terms of priority with which the test cases need to be executed:
Priority 1 - 350
Priority 2 - 650
Priority 3 - 200
That does look an ideal count for a small to mid sized project which comes under test execution cycles of three rounds.Here we assume that all the 1200 odd cases are part of the test execution in the regression cycle that needs to be performed after the three rounds of iteration that shall be performed . However, as in a typical scenario of test execution plan , for the first round of testing that needs to be carried out in the System Test cycle 1 - we will not have all 1200 odd test cases as part of execution cycle.Let us assume that for the first round we have close to 400 odd test cases as part of execution scope and their matrix is as under.
Priority 1 - 150
Priority 2 - 200
Priority 3 - 050
So this is how the test execution cycle need to be carried out so as to get the maximum functionality tested and here in this execution cycle we will definitely be able to add some 20% more test cases as some functionalities do get implemented as part of the change requests that were raised as part of buddy test execution.Let us not include that as part of discussion plane though they will definitely be part of the execution plan and the same needs to be redesigned.
With this execution plan in place the System Test cycle 1 gets executed. Now let us move into the next phase namely the System Test cycle 2. Suppose here in we have close to 550 odd test cases as part of execution scope and their matrix is as under.
Priority 1 - 150
Priority 2 - 300
Priority 3 - 100
Now we will execute the second test execution cycle for these test cases. But is that all we need to plan for or is it that something goes missing from our end in this case ? Yeah so here is something more to it.We must have logged some bugs as part of the earlier cycle, so that needs to be re-tested. But does our job as test execution cycle owner gets over. No.What is it that we need to look out for in this case. We need to have a matrix in place that helps us to assist the areas of impact due to the bug fixes introduced into the system. How do we track that ? There comes something called bi-directional traceability matrix with the additional feature of vertical traceability getting added and tracked. Is that something you have not heard for ? Ok I will being throwing some light on this black hole. Lets see how much you can pierce through the same.
Bi-directional traceability is simple as it gives you the opportunity to track the business requirement getting failed due to a bug logged. Now just imagine what vertical traceability would refer to?Yeah, It is simple, it refers a linkage from one Business requirement to another business requirement. That is which requirement might impact which other requirement.That makes test execution interesting or rather more complex than what many others feel about it !!! Do you now agree that testing a product is far more difficult than developing it.
So you end up testing all the impacted areas as part of this test execution cycle.
I have seen many projects where in the test lead does not break his/her head much into defining the impacted areas and maintaining traceability matrix , but they just end up executing all the test cases from execution cycle 1 also in the execution cycle 2.That gives clear indication of DULL work methodology and lack of SMART work , and making the job of complete test team monotonous.
Something that comes as a stumbling block typically in a Service industry based project is insufficient clarity on the documented business requirements. So in cases such as these I have my clarity in terms of Test cases and I maintain a vertical traceability on the test cases, so I get clear vision on which test case when failed and fixed might impact which other test cases.I hope that sounds effective solution model to the problem we geared up for in the current context.
Now you might have understood how the test execution becomes more challenging as the product matures more in subsequent cycles. Let us move into the next phase namely the System Test cycle 3.So this is the last phase of major development cycle and with this we will come to and end to the functionality development. Here we are left with close to 200 odd test cases as part of execution scope and their matrix is as under.
Priority 1 - 050
Priority 2 - 100
Priority 3 - 050
Now we will execute the final test execution cycle for these test cases and also plan for the bugs retesting and impact areas testing just the way we did in previous cycles.
Once the same is done regression test needs to be performed.One thing comes here at the back of my mind is does regression need to be performed or is it that testing the impacted areas for the bugs logged in the test execution cycle three would be sufficient. So two major points comes into my mind from the experience I have had in terms of the way things get handled in the industry. First point is of course the bugs logged in the cycle 3 needs to be tested. Second point is the major one to look out for and plan effectively to reduce the risks getting created due to this.On almost all occasions development team do end up in not being able to fix the bugs logged in previous cycles in subsequent cycles. How do they handle this miss from their end ? They basically increase the stack rank and keep the bugs low on priority , to which test team also need to confirm from their end as to whether delaying that fix is a risk to the product quality or it can be fixed later on .So test team here bring the vertical traceability matrix and clarify on how many requirements are impacted due the logged bug and to what extent the bug - fix for the same can be delayed is feasible. So for all these type of bugs that are lower on stack rank become part of fixing prior to the regression cycle. Also there might be some misses in terms of the test execution cycle which can also be re-addressed in the regression cycle. So all the test cases become part of the regression cycle execution.Based on the joint decision made by the entire project delivery team and the product owner, the project is then released into the pre production environment where in the UAT is performed. Since the test execution has been modeled well enough no major issue is bound to come and the project deployed into production environment which can then be released after a small round of sanity testing with some live data.
I would very soon come up with some screenshots on how these modelling can be achieved in a test management tool. I have MTM with me and I will just show up how organising things in it helps us keep track things in a nice manner.For the time being keep browsing for more on this blog, It loves quality assured nature in you. :)
Post a Comment