Automation Testing Framework Approach:
A framework approach to create an Automated Regression Test Bed has many benefits over the traditional simple capture/playback or custom scripts methodology. However evolving a keyword driven framework approach for automating the regression test bed of scripts, may take time and is generally not recommended if at all any major changes are expected to be met up in near or even near far future.
How to do Automation Testing :
• Reuse of Tests : When a certain set of test scenarios require frequent execution it makes more sense to automate them and reap the benefits of unattended automated execution of test scripts.
• Another example will be in cases where a single script needs to be executed with multiple data sets. In such cases the effort to automate a single script and running it with multiple data sets is far less than the manual execution effort for all those data sets.
• Time Save : Running unattended automated test scripts saves human time as well as machine time than executing scripts manually.
• Better use of people : While automated scripts are running unattended on machines, testers can do more useful tasks.
• Cost Saving : On test engagements requiring a lot of regression testing, usage of automated testing reduces the people count and time requirement to complete the engagement and helps reduce the costs.
• Machines are more reliable than humans : Great confidence will be gained when a system is released, when we use automated testing . With some approach instead fo just record and is playback , it becomes very easy easy to maintain and update the scripts with newer releases or changes done to the application. Also the degree of scripting gets minimized as the resources working on the automation scripting mature to successfully understand and develop some set of library for their reusability.
• Selective testing : There should be an easy way to only test the breadth of the application without doing a lot of depth testing. This is to be used for stability testing of an application • There should be an easy way to select test scenarios at a more granular level in order set up more targeted regression.
• Recovery : If test script execution is stopped at any moment for any reason, the test suite should be able to resume testing from that point at any time later. We should not need to re-test functionality that had passed earlier.
• Scalability : It should be possible to execute tests from different machines at the same time in order to gain time efficiencies.
• Inter-environment portability : The scripts should run on any given test environment without any code change (technical intervention).
• Usability : With minimal amount of training, anyone on the team must be able to execute the automated suite . The script must run without any manual intervention.
• Reporting : Test results must be reported in a way that lets a user know the status of each test script easily. Failed scripts must be accompanied by appropriate error messages and screenshot(s) that helps the user understand the exact problem. As far as possible, the information generated by logs of failed scripts must be necessary and sufficient for the user to log a defect.
Technical Requirements For Automation Testing :
• Data abstraction : The test data needed to execute scripts must be completely isolated from the actual script code. All data needed by scripts must be stored in data sheets, and the process for updating the data sheet must be documented.
• Maintainability : The automated suite should easily be maintainable for updates to application’s functionality. The following items detail out the technical requirements that will help achieve maintainability.
• Separation of concerns : Distinct layers in architecture must address concerns at each layer.
a. In particular : Logging, reporting, exception handling etc must be handled in a separate layer.
b. Core functionality for testing must be handled in a separate layer . This will help the script author to focus on the core task of writing a script that addresses the business functionality; while delegating exception handling and other non-core tasks to other layers.
• Modular code : Code written should be modular, so that:
a. Reuse is possible – if a functionality has already been coded, the functions written should be generic enough to be reusable in a different context.
b. Centralized changes – if a functionality changes, the script author must be able to make change to one central function that should suffice for the whole suite.
• Portability : The suite must be portable to:
a. A different operating system (e.g. Windows 2003) without significant changes.
b. A different tool (such as Quick Test Pro) without significant changes.
Various Automation approaches : The various approaches to Test Automation are listed below as per the evolution of their methodology and the market adoptability.
i. Generation 1: Record and Replay
ii. Generation 2: Custom Scripts
iii. Generation 3: Data Driven Testing
iv. Generation 4: Keyword Driven Framework
v. Generation 5: Business Process Driven
Generation 1: Record and Replay :
a. Interacts with the application in a similar fashion as an end user.
b. Usually involve writing an automated script per test script.
c. Usually involve unrepeatable and undefined processes leading to high maintenance costs.
Generation 2: Custom Scripts :
a. Enhancing record and replay scripts to work more than once.
b. Recording tools usually generate scripts based on the recorded user actions and allow the test developer to enhance the script for expanded functionality.
c. Libraries of common functions get developed.
d. Reuse of code starts happening
Generation 3: Data Driven Testing :
a. Application input data are extracted from the test script for maintainability and expansion of scope
b. Validation data are maintained separately and are compared with the input data
c. Data is stored in flat files, spreadsheets, or databases depending upon the data complexity and the extent to which it needs to be accessed by test scripts
Generation 4: Keyword Driven Framework :
a. Overall approach and strategy is defined for all test processes.
b. Enables writing tests in a more abstract manner using Keyword approach.
c. Enables a non-technical subject matter expert / business user to design tests without ever touching a script
d. Allows automation to be started earlier in the SDLC even before a stable application is delivered for testing thus providing more time for testing.
e. Allows a high degree of reuse among all artifacts: data, code, input files, comparison data, error handling, and reporting mechanisms
Keyword Driven Framework Approach For Automation Testing :
This solution approach is motivated by efficient usage of people by using the right level of people skills to perform the various testing tasks.
• Test design is treated as a separate but integrated activity from test automation, so we can assign differently skilled people to each activity. Test design requires knowledge of the application domain as well as test design techniques, but does not require familiarity with programming.
• Test automation requires familiarity with programming and optimization techniques.From the SME or business user perspective, test cases should be designed at a higher level of abstraction with little or no knowledge required on the tool specific programming language. To cater to this need, we used the spreadsheet for test case design in which the test case designer would write the test case in a high level language. A test case processor implemented within the framework will then interpret and process the spreadsheet. This high-level test design language contains keywords and optional data needed for each keyword. The keywords are the test steps within the test case and generally encapsulate a business event or activity. The keywords are atomic in structure and can be reused in multiple test cases. The optional data provided with a keyword may be a specific test case input or expected results. Keywords may be specific to software under test and may vary from one another. The test case processor within the framework comprises of an interpreter containing general functions to read the test cases specified in the spreadsheet. On encountering a keyword, the function implementing that specific Keyword is called and execution results will be written into a test results report.
For gaining knowledge on Automation Testing using various tools such as QTP VSTS :
Automation Testing On Share Using QTP VSTS
For gaining more insights in the automation using QTP log on to below url :
Automation Testing Using QTP
A framework approach to create an Automated Regression Test Bed has many benefits over the traditional simple capture/playback or custom scripts methodology. However evolving a keyword driven framework approach for automating the regression test bed of scripts, may take time and is generally not recommended if at all any major changes are expected to be met up in near or even near far future.
How to do Automation Testing :
• Reuse of Tests : When a certain set of test scenarios require frequent execution it makes more sense to automate them and reap the benefits of unattended automated execution of test scripts.
• Another example will be in cases where a single script needs to be executed with multiple data sets. In such cases the effort to automate a single script and running it with multiple data sets is far less than the manual execution effort for all those data sets.
• Time Save : Running unattended automated test scripts saves human time as well as machine time than executing scripts manually.
• Better use of people : While automated scripts are running unattended on machines, testers can do more useful tasks.
• Cost Saving : On test engagements requiring a lot of regression testing, usage of automated testing reduces the people count and time requirement to complete the engagement and helps reduce the costs.
• Machines are more reliable than humans : Great confidence will be gained when a system is released, when we use automated testing . With some approach instead fo just record and is playback , it becomes very easy easy to maintain and update the scripts with newer releases or changes done to the application. Also the degree of scripting gets minimized as the resources working on the automation scripting mature to successfully understand and develop some set of library for their reusability.
• Selective testing : There should be an easy way to only test the breadth of the application without doing a lot of depth testing. This is to be used for stability testing of an application • There should be an easy way to select test scenarios at a more granular level in order set up more targeted regression.
• Recovery : If test script execution is stopped at any moment for any reason, the test suite should be able to resume testing from that point at any time later. We should not need to re-test functionality that had passed earlier.
• Scalability : It should be possible to execute tests from different machines at the same time in order to gain time efficiencies.
• Inter-environment portability : The scripts should run on any given test environment without any code change (technical intervention).
• Usability : With minimal amount of training, anyone on the team must be able to execute the automated suite . The script must run without any manual intervention.
• Reporting : Test results must be reported in a way that lets a user know the status of each test script easily. Failed scripts must be accompanied by appropriate error messages and screenshot(s) that helps the user understand the exact problem. As far as possible, the information generated by logs of failed scripts must be necessary and sufficient for the user to log a defect.
Technical Requirements For Automation Testing :
• Data abstraction : The test data needed to execute scripts must be completely isolated from the actual script code. All data needed by scripts must be stored in data sheets, and the process for updating the data sheet must be documented.
• Maintainability : The automated suite should easily be maintainable for updates to application’s functionality. The following items detail out the technical requirements that will help achieve maintainability.
• Separation of concerns : Distinct layers in architecture must address concerns at each layer.
a. In particular : Logging, reporting, exception handling etc must be handled in a separate layer.
b. Core functionality for testing must be handled in a separate layer . This will help the script author to focus on the core task of writing a script that addresses the business functionality; while delegating exception handling and other non-core tasks to other layers.
• Modular code : Code written should be modular, so that:
a. Reuse is possible – if a functionality has already been coded, the functions written should be generic enough to be reusable in a different context.
b. Centralized changes – if a functionality changes, the script author must be able to make change to one central function that should suffice for the whole suite.
• Portability : The suite must be portable to:
a. A different operating system (e.g. Windows 2003) without significant changes.
b. A different tool (such as Quick Test Pro) without significant changes.
Various Automation approaches : The various approaches to Test Automation are listed below as per the evolution of their methodology and the market adoptability.
i. Generation 1: Record and Replay
ii. Generation 2: Custom Scripts
iii. Generation 3: Data Driven Testing
iv. Generation 4: Keyword Driven Framework
v. Generation 5: Business Process Driven
Generation 1: Record and Replay :
a. Interacts with the application in a similar fashion as an end user.
b. Usually involve writing an automated script per test script.
c. Usually involve unrepeatable and undefined processes leading to high maintenance costs.
Generation 2: Custom Scripts :
a. Enhancing record and replay scripts to work more than once.
b. Recording tools usually generate scripts based on the recorded user actions and allow the test developer to enhance the script for expanded functionality.
c. Libraries of common functions get developed.
d. Reuse of code starts happening
Generation 3: Data Driven Testing :
a. Application input data are extracted from the test script for maintainability and expansion of scope
b. Validation data are maintained separately and are compared with the input data
c. Data is stored in flat files, spreadsheets, or databases depending upon the data complexity and the extent to which it needs to be accessed by test scripts
Generation 4: Keyword Driven Framework :
a. Overall approach and strategy is defined for all test processes.
b. Enables writing tests in a more abstract manner using Keyword approach.
c. Enables a non-technical subject matter expert / business user to design tests without ever touching a script
d. Allows automation to be started earlier in the SDLC even before a stable application is delivered for testing thus providing more time for testing.
e. Allows a high degree of reuse among all artifacts: data, code, input files, comparison data, error handling, and reporting mechanisms
Keyword Driven Framework Approach For Automation Testing :
This solution approach is motivated by efficient usage of people by using the right level of people skills to perform the various testing tasks.
• Test design is treated as a separate but integrated activity from test automation, so we can assign differently skilled people to each activity. Test design requires knowledge of the application domain as well as test design techniques, but does not require familiarity with programming.
• Test automation requires familiarity with programming and optimization techniques.From the SME or business user perspective, test cases should be designed at a higher level of abstraction with little or no knowledge required on the tool specific programming language. To cater to this need, we used the spreadsheet for test case design in which the test case designer would write the test case in a high level language. A test case processor implemented within the framework will then interpret and process the spreadsheet. This high-level test design language contains keywords and optional data needed for each keyword. The keywords are the test steps within the test case and generally encapsulate a business event or activity. The keywords are atomic in structure and can be reused in multiple test cases. The optional data provided with a keyword may be a specific test case input or expected results. Keywords may be specific to software under test and may vary from one another. The test case processor within the framework comprises of an interpreter containing general functions to read the test cases specified in the spreadsheet. On encountering a keyword, the function implementing that specific Keyword is called and execution results will be written into a test results report.
For gaining knowledge on Automation Testing using various tools such as QTP VSTS :
Automation Testing On Share Using QTP VSTS
For gaining more insights in the automation using QTP log on to below url :
Automation Testing Using QTP