Automation and Virtualization
Contents
1. Copyright 2011 Fiserv Inc All rights reserved All brand names and trademarks belong to their respective owners The Challenges in Software Testing Automation and Virtualization 7 Virtualization Virtualization refers to an act of creating a virtual version of something including but not limited to a virtual hardware platform operating system OS storage device or network resources It refers to creation of a virtual machine just like the real one but not the original operating system In order to achieve virtualization some key factors should be kept in mind which are as follows Ensure high availability of environment Maximize Resources Improve Performance Security eo However while achieving virtualization various challenges are faced Some of the most common challenges faced while achieving virtualization are as follows 1 VM Slowness While working on the virtual machines few things which we experience is slowness of the applications tools or basic day to day software s which we use Solution e Adjust the visual effects settings performance settings of the machine to Adjust it for best performance e Changed the virtual memory setting to set it on the basis of System managed size e Avoid using any wallpaper screen saver on both Local and VM workstation e Changed the display properties of both local and VM workstations by either selecting the color size to be minimum i e 16 bit e Run an2. QTP to automate the test scenarios defined in the regression test pack This framework mainly focuses on re usability of automation code and automated test scripts which reduces the maintenance effort This translates to a significant savings over the course of time The framework architecture diagram is shown below Company Confidential Copyright 2011 Fiserv Inc All rights reserved All brand names and trademarks belong to their respective owners The Challenges in Software Testing Automation and Virtualization 11 Recovery is Driver an Test a lt I Test Suite lt gt sera nation EA if QTP Object cma ee Test Data Reporting pmm Local Data est nana aa A Variables Data Sheets The driver script written in VB Script invokes the execution It launches QTP which loads the test scripts marked for execution in Cycle Driver Excel sheet These test scripts load the corresponding object repository shared and or local test data global variables and or data sheets and calls the re usable functions automation code in VB Script from the function library It then performs the appropriate operations on the Application Under Test AUT as per the business flow At the end of test execution the framework generates a user friendly test report in Excel HTML pdf format which contains total execution time total test scenarios executed number of test scenarios passed failed environment details machine name on which test
3. box Click the Change button in the Virtual memory box Select System managed size Click OK gt OK gt OK and close Window Avoid bringing up any Application on the Local Workstation when on a call Avoid using a background and screen saver on VM and local workstation desktop Verify display quality is set to 16 bit on the virtual machine and local workstation desktop Other local workstation and virtual machine considerations Company Confidential Copyright 2011 Fiserv Inc All rights reserved All brand names and trademarks belong to their respective owners The Challenges in Software Testing Automation and Virtualization 9 I Machine Both If using Vista or Windows 7 disable side bar options li Machine Local Workstation Verify Windows auto updates are set to non hours lil Machine Local Workstation Verify Update cycle of security software is set to non business hours iV Machine Both Do not run streaming internet media from either the local workstation or the virtual machine this includes internet radio television excessive news feeds 8 Verify all local workstation NIC drivers are current 9 Disconnect Virtual Machine Make sure to disconnect your Virtual Workstation by the end of the work day Go to Start gt Disconnect 10 Run and Schedule Disk Defragmenter on Virtual Machine Run the Disk Defragmenter to reorganize your disk space Then schedule the Disk Defragmenter to run on a monthly basis 11 Run and Sc
4. cause object identification issues gt To resolve the banking domain knowledge and vb script coding skill proper 2 week training were planned with the Business Analyst and Quick Test professional Tool training were given gt Framework enhanced first to meet the business partner reporting and scenario description requirement Took 20 hours to make changes in framework Converting HTML reports to PDF and protected from any further changes Company Confidential Copyright 2011 Fiserv Inc All rights reserved All brand names and trademarks belong to their respective owners The Challenges in Software Testing Automation and Virtualization 10 gt Reusable functionality identified and functions are created and shared object repository created Team then started preparing the scenario automation script Automation script verification point s screens are then verified by Business analyst and if some changes in functionality or workflow required is implemented To avoid script changes scenario manual workflow is verified first gt For Environment configuration setting and setup changes challenges in first cycle we prepared the issue log related to configurations and setup with proposed solution is maintained which further set in other environment comes For this shakedown activity we take 4 hours to setup to avoid execution roadblock for new environment Also for the scenario which are require any data setup we upfront created pre requi
5. 22 s 20u2220u2u20u2uu0u2uu0unnnnnnnnnnn 6 Challenge 7 Low Return on Investment ssssss222222 222222222220u2220u2020u2 2u20unnnnnnn 6 VIFULI AN ZATION cisien REAA REE 7 Virtual MaCHINGe OptiMizati li aicrrrrannsnannain aaa AARAA RNANA KNARA 8 Implementation of challenges and solution in Existing Project ssssssssnnsnnnnnnnnnnn 9 Test Automation Framework Overview Basic Approach to Automation s sss 10 Salient Features amp Benefits wesisisisiciswectesenmascrecsansweewwnssencswesdeseewinewoevieierstwwescwnriemews 11 CONCUSSIONE sorrir jorin inrite noO EENES EEEE i 12 ACKNOWledgementS sirrien painan ES RERAN KAANAA 12 Reference Materials sscosssanscnscsncscosdswudcnsessnscnsassudeseswencnssesmacnnasecescsdeewaeessssianeiaewdes 12 ADOUL FISGEY sonanu a AA a aaa 12 Au thor s Biography siscsiuatesscistcccnacsnvevstcnsecsmicinesshacdaecnerdeweussandeerssaceseensecsnveusevunvenxes 13 Company Confidential Copyright 2011 Fiserv Inc All rights reserved All brand names and trademarks belong to their respective owners The Challenges in Software Testing Automation and Virtualization 3 Abstract The perpetual demand to cater to reduce the cost invariably leads Quality Assurance QA team to test automation Most organizations have adopted test automation only in large programs Typically in an organization large programs with sizeable budget invest in automation since they are able to demonstrate ROI an
6. I Verify the Visual Effects and Advanced Virtual Machine Performance options are set correctly Performance Options EEG irtual Memory 2 E Visual Effects Advanced Data Execution Prevention l ae Drive Volume Label Paging File Size MB Select the settings you want to use for the appearance and performance of Windows on this computer Let Windows choose what s best for my computer Adjust for best appearance Custom Paging File size For selected drive Drive Space available 7951 MB O Animate windows when minimizing and maximizing C Fade or slide menus into view C Fade or slide ToolTips into view C Fade out menu items after clicking C show shadows under menus O Show shadows under mouse pointer C Show translucent selection rectangle J show window contents while dragging Q Slide open combo boxes J Slide taskbar buttons T71 Smooth edges of screen Fonts z Custor size Initial size MB 2046 Maxirnum size ME 4092 i No paging file Total paging file size For all drives Minimum allowed z2 MB Recommended 2998 MB Currently allocated 2046 MB Go to Start gt My Computer select gt Properties Click the Advanced tab select gt Performance Settings button On the Visual Effects tab select gt Adjust for Best Performance On the Advanced tab select Programs under the Process Scheduling box On the Advanced tab select Programs under the Memory usage
7. ce implementation estimation how to approach new assignments She is QTP Six Sigma Green Belt and Load Runner certified professional She has earned Bachelors of Engineering and currently she is pursuing an Executive Post Graduate Diploma in International Business from Indian Institute of Foreign Trade Surender Singh Co Author Mr Surender Singh surender singh fiserv com has around 1 year of experience in Software Testing and working as Associate Software Engineer in Fiserv India Noida His core competency is automation testing and focuses on other areas like performing feasibility analysis of different applications for automation continuous improvement in automation framework quality control activities automation knowledge sharing conduct QTP trainings He is a QTP certified professional He basically comes from a development background and has done 6 months DAC Course Diploma in Advance Computing from C DAC Pune Government Institute for Research and Development in IT Company Confidential Copyright 2011 Fiserv Inc All rights reserved All brand names and trademarks belong to their respective owners
8. d other benefits in more compelling fashion These programs tend to focus on quick deployment and not on reuse or long term development of automation processes Over the period of time QA team struggle to sustain the same benefits that automation previously provided Software Testing is critical to ensure that the existing functionality of software application is not impaired by modification and upgrades as independent testing is still evolving and the software application upgrade is dynamic and challenging Also creative approach is required to automate the software successfully and to gain the long term benefits The key area of concern is large project for automation and cross platform with frequent software upgrades which require quick turnaround time The focused approach to automation defined here will enables testing organization to cope with the challenges faced in each test automation life cycle Objective Organizations are generally of the view that test automation should be used for large projects only as it reduces timelines and improves quality It is true that automation reduces timeline and improves quality but to make good use of automation it is required to follow some processes and strategies Some of the roadblocks and strategies involved in automations are discussed in this document Roadblocks deter some organization from investing in test automation Also it is a known fact that test automation requires considerable bud
9. d schedule Disk Defragmenter on the Virtual machines e Run and schedule disk cleanups 2 Dependency on third party tools like VMWare or Junos for VPN to connect to the virtual machine 3 Third party tools have sometimes impact dependency conflict on tools which we currently use ex VMWare view client software is a java based software so installing Java Add In in QTP causes conflicts 4 Virtual machine causes additional space infrastructure requirements Solutions to optimize the virtual machine are caching defragmentation etc 5 Network Limits Implementation When connected with the VPN Junos Pulse the rest the network goes down i e it is not integrated with rest of the network 6 Transfer of data from the virtual machine to the local workstation is not possible Company Confidential Copyright 2011 Fiserv Inc All rights reserved All brand names and trademarks belong to their respective owners The Challenges in Software Testing Automation and Virtualization 8 Virtual Machine Optimization Use Webmail to Access Outlook on Virtual Workstation on a limited basis Only Start Web Applications on the Virtual Machine from icons established on the desktop Avoid running Web Applications like Facets and Knowledge Central from Team Central Establish application icons on the Virtual Machine desktop and always access the application using the desktop icons 3 Visual Effects and Advanced Virtual Machine Performance viii U
10. eadline e Slow learning curve in business as well as tools e More script defects leads failure to uncover application defects e Customer lost confidence on Test Automation Solution Resource Selection e Efficient Knowledge Transition process should be in place e Backup plan for each resource e Manual testing team should also be involved in the automation aspects e Form tool excellence committee in team for R amp D a food for knowledge update e Minimize resource attrition and retain resources by motivations awards and etc e Weekly Technical Issue meeting to be conducted Challenge 5 Test Estimation Estimation based on the test steps in the Manual test cases Complexity was derived based on the number of steps in the manual test cases Control Factors were calculated based on experiences not by statistically proved data Test Data creation effort was not accounted in the estimation VV V Y Solution Test Estimation gt Follow Test Point Analysis estimation technique gt Key points to be considered for estimation e Size of the Application e Test Strategy Company Confidential Copyright 2011 Fiserv Inc All rights reserved All brand names and trademarks belong to their respective owners The Challenges in Software Testing Automation and Virtualization 6 e Productivity gt Derive Test Points from Functional Points gt More factors that also to be considered e Environment Factor e Quality Factor g
11. execution was done executed by operating system verification point screen shot etc gt A oon ot gt oO LL D Application Under Test 4 Salient Features amp Benefits In addition to standard features such as performing operations and verifications on the test objects this hybrid framework offers other sophisticated features like 1 Independent test Architecture Wise use of Descriptive programming and Vbscript coding used 3 Flexibility to create application specific modular functions operations or task that cannot be performed by library functions for example composite test objects 4 Ability to test variety of applications like web windows based and AS 400 IBM Mainframe 5 Multi language support 6 Optimum utilization of the test automation tool 7 Simple Test Design a non technical or a novice user tester can effectively create test scripts with minimal training or mentoring effort 8 Configurable framework settings 9 Create reusable automated test scripts 10 Parameterization of test scripts iterate for same or different data 11 Maintenance amp Expandability is easier and quicker 12 No database required 13 Smart exception handling 14 Unattended execution attained a Company Confidential Copyright 2011 Fiserv Inc All rights reserved All brand names and trademarks belong to their respective owners The Challenges in Software Testing Automation and V
12. fiserv 13 Annual International Software Testing Conference Paper on The Challenges of Software Testing Automation and Virtualization Authors Primary Rajni Singh Secondary Surender Singh Email rajni singh fiserv com surender singh fiserv com Phone 91 120 409 3073 Work 91 120 409 3068 Work 91 997 179 8831 Cell 91 954 017 5120 Cell Fax 91 120 409 5001 Fiserv India Pvt Ltd 6 7 8 Floor Tower C amp D A 44 45 DLF IT Park Sector 62 Noida U P 201301 Phone No 91 120 6185000 Fax No 91 120 6185001 http www fiserv com The Challenges in Software Testing Automation and Virtualization 2 Table of Contents ADS ACE isstnsitessesdsvesnassssensuisevsosuscecessnesesnususcecesseedareasussvucssesedvensesdsasanarseusseursvusssesseus 3 OD IC CUVEE vesaciindceniairsusierens ninr ra a 3 Challenges in Automation Testing cssssccccecccccecsseneneeeeeeeeeeeeeeeeeeeneenneeeeeeeeesseeeeennanas 3 Challenge 1 Customer Expectation s 2222 2222222u2222u2220u2u20u2u20u2uuunnnnnnnnnnn 3 Challenge 2 Automation Strategy s 2 52 222222220202u20u0un0unn 4 Challenge 3 Tool Selection sssssunsnnunnnnnnnnnunnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnn 4 Challenge 4 Resource Selection crsiiscccrnsscendnaeesscsncseduatenssssvarecateesvanureveussedeeennenwews 5 Challenge 5 Test EStUMatjO Murire Aa 5 Challenge 6 Technical ISSUES sssssuss22
13. gets and efforts This paper provides an insight of best practices to be followed and strategies to be adopted in order to get good returns from test automation This paper also provides an end to end test automation framework that helps overcome the key challenges It also covers the issues faced in automation testing on virtual machines and benefits of virtualization with key solutions to the roadblocks Challenges in Automation Testing Few challenges in automation testing are mentioned below with the proposed solution Challenge 1 Customer Expectation gt Customer s misconceptions e Automation can be completed in less amount of time with the tool s intelligence e Tools can generate test data for automation e Scripts runs irrespective of platform environment and hardware e 100 unattended script execution Company Confidential Copyright 2011 Fiserv Inc All rights reserved All brand names and trademarks belong to their respective owners The Challenges in Software Testing Automation and Virtualization 4 e Scripts capture all defects gt Customer are not aware of e Principles and Behavior of testing tool e Technology used in tools Solution Customer Expectation gt Educate customer about e True benefits of automated testing such as Reliable Repeatable Saves time Programmable etc e Testing tools and their pros and cons e Test automation framework and ROI e Test script maintenance effort Challenge 2 Automati
14. hedule Disk Cleanup on Virtual Machine Implementation of challenges and solution in Existing Project Our Project is for Tesco bank for Signature Europe requirement of Project that we need to automate around 460 business scenarios in 3 months of time with complete regression pack execution The main objective for the Regression pack is that it should work for different environment and result can be verified and compared with each next release Reports Should save in secured manner so that no one can do any changes Scenario Verification points and triggers points screen shot should be captured and verified by Business Analyst at their end Now the challenge we faced in initial phase are 1 Team Skill Both domain and technical skill was not expected for the required project Our Team comprise of 4 team members with maximum of 3 4 months of experience in banking domain as well in IT industry and one team lead with Automation expert to run this project 2 Existing Framework challenges Existing Framework does not include successful verification screen shot and also reports are not protected 3 Environment challenges Configuration settings changes and set up issues a Setup challenges include test data setup for scenario for each environment 4 Unattended Execution Automation script should execute unattended 5 Automating IBM mainframe AS 400 as IBM AS 400 is very protected mainframe type so most of the object properties are protected which
15. irtualization 12 15 Screen shots for verification point and trigger points 16 User friendly test reports in Excel format HTML and PDF format Conclusion A Reusable Automation Framework approach fits all domain automation Not only automation also the challenges in virtualization can be overcome by adopting a comprehensive strategy which focuses on initially identifies the challenges and applies the proposed solution Deployment of Reusable Automation framework in different business units within Fiserv Global Services provided significant benefits in terms of savings in test designing and maintenance test execution cycle time component reusability and framework extensibility Test automation helped Fiserv QA teams to re allocate resources to perform specialized testing operations In turn productivity of resources increased along with enhanced software quality Some of the take away from the above explained challenges and comprehensive strategy are 1 Automation and Virtualization challenges identified at early stage with proposed solution which help us to decide our approach while initiating Automation and Virtualization in Project 2 Ready to use Framework 3 Secured and protected Reports in pdf format 4 Easy to maintain and create scripts 5 Reports can be used for Application understanding document Acknowledgements We would like to thank everyone who supported us directly or indirectly We would like to express our gra
16. on Strategy gt Ad hoc test automation strategy e One Test Case One Script Automation approach Disadvantage e High maintenance cost of large script suits e Result Analysis and reporting takes huge time for a releases gt Lack of process in identifying tests to be automated e No guidelines for test case identification e Inefficient process in place Solution Automation Strategy e Defined Test Automation Process implementation e Requirement Driven approach for test automation Challenge 3 Tool Selection gt Compatibility with AUT gt Cost and performance gt Lack of tool support from vendor e Tools not recognized Third Party objects controls in the application Company Confidential Copyright 2011 Fiserv Inc All rights reserved All brand names and trademarks belong to their respective owners The Challenges in Software Testing Automation and Virtualization 5 Solution Tool Selection gt Tool Selection Team should be formed before selecting a tool gt Automation team should ensures e Analysis of backward compatibility of various versions e Tool s incapability and defects e Continuous evaluation of Tool upgrades e Compatibility with configuration management tools gt 24 X 7 Customer Support from Vendor Challenge 4 Resource Selection gt Lack of skilled resources gt Use of Sub contractors in team gt Fresh resources are inducted e Rigorous training is executed to cope up with project d
17. site test data script for new environment execution gt As best practices we maintain Issue Log update traceability matrix Automation Run book Failure analysis is easier with pdf report document daily status update with business partner and defect are maintained in Quality center gt Unattended Execution was the biggest challenge e Banking application and office premises security issue for system unlock Unlock code written so that closed screen can take screen shot and system does not go to sleep e Pcomb process is required to successfully verify most of the scenarios Automated scripts were dived before and after pcomb run steps and well maintained in batch in cycle driver e Recovery scenario reduces the performance of scripts as well processes are slow Functions created and called where ever recovery is required e System Performance for large batches execution To avoid the low system performance due to cookies and temp files created while execution simple clear file and cache delete code written and after batch of 10 or any intermittent error this code execute and resume to execution Also on AS 400 reusable login and purge code written to have secure login each time it fails to connect Test Automation Framework Overview Basic Approach to Automation A hybrid framework which incorporates the best practices of Test Library Test Modularity Data Driven and Descriptive Programming approaches is developed which will be used with
18. t Represent effort break up in terms of percentage Challenge 6 Technical Issues VV VV VV VV VV V Y Tools not recognized Third Party objects controls in the application On the fly GUI object object property change Objects names are not meaningful and unique Color Code validation is not successful Graphical outputs and Image validations are not successful Multi browser and multi platform validations are not successful Critical functionality in the application were not effectively automated Additional manual steps required to carry out the automation Solution Technical Issue Identify non recognizable objects in early phases of automation Work collaboratively with Design and Development team Define Scope in Scope Out in Test Automation Plan Create utility scripts to support the main automation scripts through R amp D Challenge 7 Low Return on Investment VV VV Y Schedule and Effort over run due to higher degree of script maintenance Negative impact on SLA compliance for script execution Higher degree of challenges in unattended execution Frequent functional change Application Performance Instability Solution Increase in Return on Investment Implement Process for Test Automation Provide detailed efforts to develop automation script Implement Automation Frame Work Plan and organized scripts execution for each build VV V WV e Risk Based Execution e Maximize unattended execution Company Confidential
19. titude towards Saroj Patnaik GM for reviewing our work Reference Materials 1 Automation templates and training material available on QMS intranet portal of Fiserv Global Services 2 Reusable framework user manual created by Fiserv Global Services Noida About Fiserv Fiserv Inc NASDAQ FISV is the leading global provider of information management and electronic commerce systems for the financial services industry Fiserv is trusted by more than 16 000 clients worldwide including banks credit unions thrifts of all sizes mortgage lenders and leasing companies telecommunications and utility companies brokerage and investment firms healthcare and insurance providers and retailers and municipalities Company Confidential Copyright 2011 Fiserv Inc All rights reserved All brand names and trademarks belong to their respective owners The Challenges in Software Testing Automation and Virtualization 13 Fiserv delivers financial services technology solutions in five areas of competence namely Payments Processing Services Risk amp Compliance Customer amp Channel Management and Insights amp Optimization Author s Biography Rajni Singh Author has more than 6 years of experience in Software Testing and is working as Senior Quality Analyst in Fiserv Global Services FGS Noida She has more than 6 years of experience in Information Technology with framework design in Test Automation analysis integration maintenan
Download Pdf Manuals
Related Search