TESTING REPORT: SHUNRA\STORM STX-100
Contents
1. CT Labs Tested Report Shunra Storm STX 100 The amount of data loss in the packet loss tests represented as a percentage of the total was calculated based on the amount of data sent from one test endpoint to the other and the amount of data actually received on the other end Figures and 2 below give us a graphical representation of the relationship between these measurements When plotted a direct correlation between Throughput and Transaction Rate Response Time can be seen As degradation is increased with the Shunra Storm the result is an expected drop in throughput and transaction rate and an increase in response time Throughput Transaction Rate Latency 25ms 50ms 100ms 200ms 400ms Figure 1 Throughput and Transaction Rate latency tests 80 eo 40 20 0 Latency 25ms 50ms 100ms 200ms 400ms Figure 2 Response Time latency tests Page 8 of 9 CT Labs Tested Report Shunra Storm STX 100 CTLABS LABS A B These charts make it possible to visually examine the relationship between Throughput Transaction Rate and Response Time The TESTING SERVICES cee actual data points are presented in the data at the beginning of this section About CT Labs CT Labs is an independent full Technical support The Shunra technical support staff was quite knowledgeable and capable of addressing our questions promptly service testing and product if not immediate2. Sip unit s IP address updating licenses upgrading firmware and so on The Shunra Storm hardware is a 3U rack mountable unit The chassis allows for port upgrades via proprietary blades hosting the desired interface type The unit that we received for this evaluation came equipped with 1 management blade and 1 STX 100TX blade 4 100Mb Ethernet ports Installing the hardware was a simple matter of plugging in the power connecting our chosen console workstation into the management port via a standard RJ 45 network cable and turning it on The Shunra Storm software suite has expanded since we last evaluated it and the installation process has become arguably more sophisticated This is not necessarily a bad thing and regardless it works well as long as you read enough of the documentation to know what you should be installing at this point Page 4 of 9 CT Labs Tested Report Shunra Storm STX 100 Documentation We found the Shunra Storm documentation to be extensive and very useful in getting started using each feature A set of 3 small binders is included boasting nearly 400 pages total The User Manual comprises two of the binders the first covering installation and setup of the Shunra Storm hardware and included software the second covering the Web based GUI and other monitoring tools and the third is devoted to the new StormTraffic and its integrated load testing tools The documentation is well complemented wit
3. TCP IP Endpoint 1 Endpoint 2 192 168 1 98 192 168 1 99 NetIQ Chariot v4 3 NetIQ Chariot v4 3 StormConsole v3 1 Testing Setup Description For our tests we connected two NetIQ Chariot endpoints to the Shunra Storm via RJ 45 Ethernet cables in a manner similar to hooking up to a hub The workstation we used as the StormConsole in our test setup was similarly connected After we installed all necessary Shunra Storm software on the console we drew a simple network diagram by using the Shunra Storm s add in Visio objects This diagram consisted of the WAN cloud and two endpoints with appropriate IP addresses We then used this diagram for all the tests performed and changed the WAN conditions via the WAN cloud properties to define varying levels of packet loss and latency The NetIQ Chariot product provides a capability to generate scripted IP data streams that allows the accurate measurement of data throughput and other IP transmission performance parameters Page 2 of 9 CT Labs Testing Report Shunra Storm STX 100 Summary of Tests Performed Ethernet Degradation Test CT Labs utilized a file send script included with the NetIQ Chariot software for the clear test as well as the latency tests This script sends a 1MB file from one test endpoint to the other in a loop for the specified test length in this case 30 minutes Performance data such as response time throughput and transaction rate were recorded
4. for the duration of the test run This test was executed several times each time applying a different condition to the Shunra Storm between Endpoint 1 and Endpoint 2 The first test was run with emulation inactive thereby simulating a setup where the two workstations are linked by just a hub which should result in no errors and negligible latency We then ran several with emulation activated to simulate different WAN conditions Performance data was then gathered for off line analysis The length of time that each test was run was 30 minutes To test packet loss emulation CT Labs utilized a NetIQ Chariot script that simulates a Real Media streaming audio video session and records performance data such as throughput and data loss These tests were also executed for 30 minutes each Please note that the purpose of these scenarios is not to load or stress test the maximum performance of the Shunra Storm but rather exemplify the accuracy of the Shunra Storm s impairment algorithms These tests set out to validate that when a user specifies 2 random packet loss for example he or she can actually trust that the Shunra Storm is providing accurate emulation of an environment with 2 packet loss Page 3 of 9 CT Labs Tested Report Shunra Storm STX 100 CT LABS CT Labs Test Results and Findings TESTING SERVICES Installation We found the installation process for the Shunra Storm to be very straightforward and did not encounter p any technical
5. problems Two CDs are included with the Evaluation Scores Shunra Storm appliance The first CD in the set contains a copy of Microsoft Visio 2002 Standard which should be installed before anything else The second CD in the set contains all additional ee Shunra Storm related add ons When the second CD is inserted Test Category Ease of installation the main Setup program is launched and displays a list of all components that can be installed individually with a very brief description of the item selected We needed to consult the Graphical user i 9 5 documentation regarding what some of the additional components interface ease of use ibe Sidi ce did as the short descriptions here didn t quite make this clear to us Product features right off After getting a clearer picture of what each item was and Product performance which were required for use we went ahead and installed everything Using a top down approach when installing each Technical support individual component in the list appears to be the way to run through the installation but we didn t find this stated anywhere Above scores are out of 10 For use of StormTraffic and related automation testing possible in each category components separately licensed we also installed the related agents on separate PCs Documentation The Shunra Storm also ships with a pre loaded Web interface for 3k performing rudimentary configuration tasks such as changing the
6. Our performance tests were also met with flawless execution and provided the expected results for each test condition that we exercised All in all we have been very impressed with the Shunra Storm and its capabilities and have found a permanent place for it in our lab Product Description The Shunra Storm is a powerful WAN emulation tool that allows you to emulate multiple WAN connections at once and recreate a multi segment enterprise WAN in a controlled lab environment Using the Shunra Storm product performance can then be tested using any combination of the Shunra Storm s highly configurable degradation parameters Application performance may be tested over the degraded WAN using captured end user traffic conditions with the new StormTraffic add on Page 1 of 9 CT Labs Tested Report Shunra Storm STX 100 The innovative and easy to use console software StormConsole can be installed on any PC running Windows 2000 Professional or Windows XP Professional The StormConsole software is a Microsoft Visio add on Microsoft Visio 2002 Standard included with the Shunra Storm which provides a number of specialized control objects for use in building your WAN environment StormConsole communicates with the Shunra Storm s hardware via TCP IP using one of the Ethernet or Gigabit Ethernet ports or the dedicated management port RJ 45 interface Testing Setup Shunra Storm Testing Setup Shunra Storm STX 100 TCP IP
7. TESTING REPORT SHUNRA STORM STX 100 CTLABS The Computer Telephony Industry s Premier Testing Service October 2003 SHUNRA Software Ltd New York NY 212 279 8895 www shunra com Product Tested Shunra Storm Model STX 100 Serial 2X023 Software Version 3 1 4 port Ethernet configuration Executive Summary As with previous models of the Shunra Storm we found the STX 100 to be an easy to use and very powerful emulation platform for performing real world WAN performance tests The STX 100 boasts a list of impressive new features and we were pleased with how friendly they are to use The STX 100 also supports up to 16 Ethernet ports 8 Gigabit ports or a combination of both Installing the equipment itself was a breeze though first time users of the Shunra Storm should probably read through the documentation related to software installation before attempting to install the software Although Shunra has done a good job of bringing all of the separate installable components to a single wizard there is a potential for confusion for those new to the Shunra Storm particularly with recognizing which components if not all should be installed There is more than enough high quality documentation to help in any area of Shunra Storm management possible from installing the software to getting started with the scripting and automation tools The documentation is well written well organized and comprehensive
8. from a single location Users can save the entire enterprise environment in a single file to be used later to recreate the same conditions when needed StormCatcher allows network performance to be monitored and recorded for historical analysis over long periods of time and easy troubleshooting Open API and scripting through Tcl and XML Easy emulation of complex networks including multicast and broadcast traffic Support for technologies such as IPv6 QoS MPLS and 802 1q Can be configured to function as a bridge or router Assists in network capacity planning Flexible hardware upgrade and maintenance Page 6 of 9 CT Labs Tested Report Shunra Storm STX 100 Product performance With our script driven Chariot data generator we ran one set of tests using the IP protocol and tracked the average throughput average response time and average transaction rate The first test condition was straight through the Shunra Storm without any induced degradation We altered the test setup on the next tests to provide conditions of latency and packet loss The results were as follows Throughput in Transaction Rate Degradation Mbps per second in seconds No 43 604 5 213 0 192 Induced Throughput Transaction Rate Response Time Latency in Mbps per second in seconds 1 122 0 134 7 478 50ms 0 602 0 072 13 926 0 312 0 037 26 853 200ms 0 159 0 019 52 628 0 080 0 010 104 375 Induced Throughput B
9. h screen shots and contains walkthrough content that walks the user step by step through setting up a sample use of each component After these follow along sections the documentation also gives the reader more detailed information including troubleshooting about each component Overall the documentation is well written and very helpful when trying out a feature for the first time User interface analysis The StormConsole is the primary interface for communication between a computer and the Shunra Storm appliance StormConsole is a Microsoft Visio add on that provides several stencil objects for recreating virtual WAN environments for testing use such as endpoints WAN clouds and gateways Microsoft Visio 2002 Standard included As in previous versions of the Shunra Storm these elements are highly configurable and very intuitively so Double clicking any of the placed objects presents the user with configurable settings for that object With these components alone it is possible to create an elaborate network emulation model for testing under realistic degraded network conditions This latest offering of the Shunra Storm is a much more sophisticated network emulation platform however As of version 3 1 the capability for load testing with specific traffic patterns has been provided with the integration of tools such as Mercury Interactive LoadRunner and Astra LoadTest StormTraffic for Web users and StormTraffic for lab auto
10. ly We have always found dealings with the analysis lab exclusively serving Shunra support team satisfactory as they are very friendly and the Converged Communications easy to reach CC and IP Telephony IPT in dustries CT Labs testing auto mation specialists not only perform a wide variety of standard test suites that validate and End of report exercise IPT and CC products but can also create custom developed tests for special testing needs Testing solutions include telephone and graphical user interface functional regression and load stress tests CT Labs prides itself on working with cutting edge IPT and CC tech nology and products The CT Labs Tested program was designed to provide independent testing and validation of products including the execution of tests that verify real world performance and ease of use Products that display the CT Labs Tested label have been tested by CT Labs and have exhibited a high level of quality in their performance and use For more information about this or other testing services contact CT Labs 916 784 7870 voice www ct labs com Page 9 of 9
11. mation through the separately licensed StormTraffic add on These additional testing tools make it possible to emulate real life end user behavior and production traffic loads StormTraffic for lab automation An automation tool that enables StormTraffic to control third party programs that support Windows command line invocation StormTraffic for web users An automation tool that enables users to create and run load tests to evaluate Web application environments and production systems Scripts may be captured or recorded in real time and modified using the include Script Modeler editor Page 5 of 9 CT Labs Tested Report Shunra Storm STX 100 Mercury Interactive LoadRunner and Astra LoadTest Automation tools from Mercury Interactive Corp that execute multiple instances of scripts to emulate an environment in which thousands of users work concurrently with client server systems Product features The Shunra Storm was verified to have the following features A fully scalable solution suite that can recreate a real world enterprise WAN environment in the lab including the production network and end user traffic Allows testing of multiple simultaneous WAN connections Easy to use GUI based on Microsoft Visio Highly configurable Visio objects for components such as endpoints gateways WAN clouds QoS emulation and frame relay emulation Central console allows users to control all StormAppliance hosts
12. ytes Percent Packet Loss in Mbps Hyves cent lost 1 0 099 23 000 000 22 772 000 0 99 0 098 23 000 000 22 525 000 2 07 3 0 097 23 000 000 22 291 000 3 08 0 096 23 000 000 22 081 000 4 00 0 095 23 000 000 21 834 000 5 07 The Shunra Storm performed accurately for both sets of tests The amount of degradation for the packet loss and latency tests evident in the test results was consistent with what was specified for each test Studies have shown TCP traffic to be extremely susceptible to significant drops in throughput as delays even in the microseconds are introduced due to its connection oriented nature Since our range of incremental latency values started at a relatively high number we see the throughput drop significantly with the first test With each subsequent test run however we see response times double as the latency values are doubled exactly what we would expect to see The amount of data successfully sent from point A to point B per second 3 Essentially the inverse of the transaction rate transaction rate is measured as transactions per second response time is measured as seconds per transaction 4 The number of transactions file transfers completed per second gt Note that the endpoint script executed for the packet loss tests was not the same as the script executed for the latency and clear tests All tests were however run for a set time of 30 minutes Page 7 of 9
Download Pdf Manuals
Related Search