Home

Server Efficiency Rating Tool (SERT) User Guide

1. C gt Click NEXT to continue after reviewing all the JVM options SENI Copyright 1989 2015 Standard Performance Evaluation Corporation SPEC All rights reserved 6 May 2015 20 of 40 Copyright O 1988 2015 SPEC Server Efficiency Rating Tool SERT User Guide 2 3 8 Choose Test Suite The Choose Test Suite panel allows the user to choose the test to run to configure key launch variables and to view configuration information about the test For a compliant run the Full Test Suite config all xml configuration must be selected Note For the purposes of informal testing the user may choose to select individual worklets For a valid run all worklets must be enabled inixi Choose Test Suite Navigator Menu Select a configuration file and then enable the tests you would like to run All tests in sai config all must be selected for valid submissions a Choose Test Configuration Full Test Suite config all xml x se Auto Ranges SUT Configuration T v Select Deselect All Minimum Test Run Time Estimate 4 hours 10 minutes 55 seconds SUT Discovery Name Enabled CPU lvi a Cc iv Test Environment cud CryptoAES v LU v PTDaemon SOR lvi XMLvalidate iv JVM Configuration Sort lvl SHA256 v JVM Review Storage v Sequential i
2. 3 6 2 Disabling the Solaris firewall The Solaris firewall may be disabled by issuing the following command as root sudo svcadm disable svc network ipfilter default 6 May 2015 30 of 40 Copyright 1988 2015 SPEC Server Efficiency Rating Tool SERT User Guide The state of the firewall can be verified using the sudo ipfstat io command When the firewall is disabled its output will look like empty list for ipfilter out empty list for ipfilter in 3 7 Lower Performance Platforms The following values in the config all xml file may be set to any integer between 1 and 10 to provide additional warmup time to allow lower performing platforms to reach steady state before measurements start e interval count id cpuWarmupIntervalCount gt 1 lt interval count gt e interval count id memCapacityWarmupIntervalCount gt 1 lt interval count gt e interval count id storageWarmupIntervalCount gt 1 lt interval count gt e interval count id hybridWarmupIntervalCount gt 1 lt interval count gt 4 Power Analyzer Range Settings Most power analyzers can be set to multiple current ranges In order to achieve accurate results the range settings on the device must be set appropriately for the load being measured Most devices support an auto ranging mode which will adjust the device automatically as the load changes however use of this mode is not allowed for measurement because it takes time for the range to adjust
3. or temperature sensor connectors if required 6 May 2015 4 of 40 Copyright 1988 2015 SPEC Server Efficiency Rating Tool SERT User Guide e Power analyzer and temperature sensor o o Note Before connecting the power analyzer and the temperature sensor to the Controller and the SUT please read the Power and Temperature Measurement Setup Guide in the Documentation directory on the SERT media carefully The latest version of this document can be found on the SPEC web site http www spec org power docs SPEC Power Measurement Setup Guide pdf This practical guide explains how to set up and run various power analyzers and temperature sensors with the SPEC PTDaemon The power analyzer is connected to the Controller and used to measure the power consumption of the SUT whilst the temperature sensor is also connected to the Controller and used to measure the ambient temperature in front of the SUT s main airflow inlet Note This release of the SERT is only intended to be used with servers running on AC power DC power measurement is currently not supported Only a power analyzer and temperature sensor from the list of accepted measurement devices http www spec org power docs SPECpower Device List html can be used with SERT The power analyzer must have been calibrated within 12 months before a SERT run is performed The following block diagram represents a simple SERT environment Controller Hi Chauffeur
4. 1 nicer ci tese rra tea nro Da ue es pna eade o Ona ges e pea aaa ra Du ee ns ora a aa Een DUs 8 2 1 1 On the SUT and Controller system cccccccccccssssssccecececsesssaesecececseseeasseeecsceeeaaeseseesceeseaasaeseeseeeseesaeeeeees 8 2 1 2 Onthe SUT only iet RR oe reete ee eee esee he fere a a aa vad ine tet eere aias 8 2 2 Invoking the Gul Rt 8 2 3 Interacting with the GUI ss 5 ice recie aea recreo AiE AEE CENNER AET EEEE AAAA RRE ee E pre Ra dau 8 2 3 1 SUE sac EE 8 2 3 2 SUT Config UrdtioDi eet eue e aai e e e a e Eu eed hne ae eee aequ ed dents 10 2 3 3 SUT IDISCOVERY AE eere etel eo nee e e bte eee iea eats ence e Ue e iaceo eae cal ocu vee e Ee Pee ne eaa PERENNE 2 3 4 Test Environment 2 3 5 PTDaemon Configuration esses e i oie ea e a i eaten sane ss esse iiA 14 2 3 6 VM COET O a a Dea eee eerte a cbe na delet ode ist a pete ire e e ER ERR 19 2 3 7 DTI IMMER T AE 20 2 3 8 Choose Test S ite 0 teer Fee rere tee peo ree ha oe gehe ee ea ae a AiE SARSA eere rase dee tia oae A aa 21 2 3 9 LaunchiTest 25 oh eo a oet E duro es RH lo coe tere ao debates gode fees dede EN de over xe ATUS IDEE 22 PESE CIS ce Ec 24 2 3 4T Help ds e a E OO RH E ea gu te Oe d E T E ERE RR 25 Platform Specific Settings die piod ied ode Rar ERIEEMAM E ORU o Pra PCR ORE AA aa Neds Ra 26 3 1 Microsoft WindOWs 1 1 eren reete noie terea rh ee erro ote EEEE VONE aaro eno E SERAN A SERS 26 3 1 1 Disa
5. Director beu SERT Workload Suite PTDaemon fo 0p o o o u q 9 fe 4 42 Temperature Sensor Power Analyzer 1 3 SUT Configuration Requirements For a compliant SERT result a basic set of hardware and software components is needed In multi node test setups all nodes must be equipped with exactly the same hardware and software components i e only homogeneous configurations are allowed 1 3 1 Platform Requirements For the list of supported platforms please see the SERT Client configuration guide http www spec org sert docs SERT JVM Options 110 html 1 3 2 Storage Requirements e Atleast one storage device 6 May 2015 5 of 40 Copyright 1988 2015 SPEC Server Efficiency Rating Tool SERT User Guide e Fora single drive a minimum of two partitions are required o OS partition s and one SERT Data partition min 20 GB e For multiple drives drives should be partitioned as follows o First drive OS partition s and one SERT Data partition min 20 GB o Any additional drive SERT Data drive entire drive capacity e For all storage devices the device cache must be disabled and each device must be configured in Write Through mode see section 3 for platform specific instructions e SERT Data partitions may only be hosted on filesystems e g ntfs ext4 zfs jfs that support direct I O e RAID arrays of more than one device are not allowed i e each individual storage
6. Select the Save check box and click Next to save the configuration values into the test environment file Cest environment xml On subsequent runs uncheck the Save box to bypass overwriting any configuration changes that might have been made If the test configuration is already correctly described in test environment xml you should deselect the Save check box to prevent this information being overwritten by SERTUI BEI SERT 1 1 0 iol xl SUT Discovery Navigator Menu SUT Info Network Address 10 0 0 101 Start System Type Windows SUT Configuration SERT Install Directory C SERT 1 1 0 SUT Discovery Test Environment PTDaemon JVM Configuration JVM Review Choose Test Suite Launch Test Results Help WARNING Discovery items are not being saved The following values have been automatically detected and will be copied into the test environment configuration Deselect the below check box if you would like to keep your test environment configuration as is or select individual items to save L Save Discovery items to Test Environment Configuration file Toggle ALL items Save Discover Item Result HardwareVendor FUJITSU v HardwareModel PRIMERGY TX1320 M1 CpuName Intel R Xeon R CPU E3 1230 v3 3 30GHz v CpuFrequencyMhz 3301 v CpuHardwareThreadsP
7. The new information is copied to the 1isteners xml file and used for subsequent SERT test runs The process to find suitable range settings for a given configuration is described in more detail in chapter 2 3 8 Choose Test Suite and chapter 4 Power Analyzer Range Settings Power analyzer specific information is given in the SPECpower Measurement Setup Guide pdf document 6 May 2015 17 of 40 Copyright 1988 2015 SPEC Server Efficiency Rating Tool SERT User Guide 2 3 5 3 Additional Power Analyzer and Temperature Sensor Device Additional device can be included by pressing the Add button 2 What type of device Power Analyzer Temperature Sensor Select the device type and modify that setting to match your device Please note to change the listing port as well one port device Example Power Analyzer added BHI SERT 1 1 0 Me xl PTDaemon Configuration Navigator Menu Power Analyzer 1 l Temperature Sensor 1 Power Analyzer 2 m ERES Listening Port e890 SUT Configuration C Remote PTDaemon IP Address 192 168 0 1 SUT Discovery r Measurement Device Dummy testing only v Test Environment UNSUPPORTED DEVICE Results will be marked invalid Analyzer Channel Number PTDaemon Interface JVM Configuration amp Serial GPIB USB Ethernet diti EA Interface Address com1 Calibration Choose Test Suite J Specify calibration date
8. own staff SERT generalists are able to do so 6 May 2015 33 of 40 Copyright 1988 2015 SPEC Server Efficiency Rating Tool SERT User Guide 6 SERT Command Line Usage 6 1 SERT Configuration and Start Procedure This section provides details to enable a successful test with a simple server configuration as an alternative to running the GUI Further sections provide additional details for some of the configuration settings that will be required in order to perform a successful test pass It is recommended to read this section as well as the SERT Design Guide to understand more of the details behind SERT and how to configure and run it Please note that in the following instructions there are typically two script file options bat for the Windows OS and sh for UNIX and Linux OSes e Configure the analyzer Edit the power analyzer PTDaemon runpower bat or runpower sh and temperature sensor PTDaemon runtemp bat or runtemp sh script files Ensure the proper communication ports and network ports are used The batch and script files are present in the SERT PTDaemon directory o For each additional power analyzer Create a copy of runpower bat or runpower sh and ensure that the proper communication ports and network ports are set o In order to analyze problems reported by PTDaemon it may be helpful to generate log files To do this edit the script files to set the LOGFILE and optionally the EXTLOG variables LOGFILE should
9. 7 Lower Performance Platforms eeeeeeeee esee ee eese eese nennen nh sn sain nn nett nt esas sanae steer nn 31 4 Power Analyzer Range Settings eee ee esee eese eene eene enn snae nuno eese so sata sensn aao 31 4 1 Automating Range Settings using the GUI ccsssssccsssssceccsssceccecssceccecssseccacssseceanssssseeess 31 4 2 Manual Range Settings Optional ccsssscccsssssceccenssccccessccccanssceccenssceccanssseseansseseeaees 32 4 3 Extracting Range Settings from Full Runs Optional eeeeeeeee eese eene eene enne 32 5 USING the Collector Scripts 2 cosaseysutcesasuvecacasvinccssaviacdsasacncdsaasabsiuasedetisiaeuaieeseetsusaasaice 32 SERT Command Line UsdQe eiii idea Nets tees ues dante attire 34 6 1 SERT Configuration and Start Procedure eee eese eee eese eene seen enn these nante nena soto sena ns 34 6 2 Generate report files with the reporter scripts ccccccsssscccsssssceccessscescsssscessesssseseessseeeees 38 7 Coefficient of Variation CV esee eiie eese esee eee ee eene nenne enn nn nione enhn unu nu un 39 7 EE ro 39 EDs SCHONE CV m 39 8 SUT Minimum Memory Requirement 4 eee eese ee esee e nee n unies annee nasa essa ases snas eaa 39 9 KNOWN ISSUES RED 40 10 Trademark eR M 40 11 Copyright NOUECE isi iioc soda cp nra lv
10. Click Next to continue Note For the purposes of informal testing the user may choose not to edit any of the test environment information provided by the default template Simply click Next to continue All status fields showing the state Complete are highlighted green to represent that all information fields have been changed from their default values It does not necessarily indicate that the information within these fields is correct and valid for publication The tester has to verify the correctness of all the information manually and make the required changes 6 May 2015 13 of 40 Copyright 1988 2015 SPEC Server Efficiency Rating Tool SERT User Guide The default parameters are defined with a leading underscore to clearly mark them as placeholder values that must be edited to produce valid results The reporter recognizes default placeholders and highlights these fields with a yellow background in the result file 2 3 5 PTDaemon Configuration The PTDaemon Configuration panel allows the user to view and edit settings associated with temperature and power instances of PTDaemon There is a tab for the Power Analyzer and for the Temperature Sensor Additional devices can be added with the Add button or removed with the Remove button this is a convenient way to retain configuration information in a lab environment where multiple devices are available LIrrprrc A 9 uugid PTDaemon Config
11. Date of last calibration yyyy mm dd 2015 03 04 Launch Test Logging C Save log output to a file outlog Results pm Save debug output to a file debug log Help Q 7 NN i i i Debug log verbosity 0 0ff 1 minimum 9 maximum 0123 45 67 89 Not Started Test Connection Add Remove SS oe Jom Copyright 1989 2015 Standard Perfomance Evaluation Corporation SPEC All rights reserved 6 May 2015 18 of 40 Copyright 1988 2015 SPEC Server Efficiency Rating Tool SERT User Guide 2 3 6 JVM Configuration The JVM Configuration panel allows the user to select the parameters of the tested SUT configuration in order to identify the corresponding Client Configuration ID in the local client configurations 110 xml file OE in JVM Configuration Navigator Menu JVM Options File Start C Users Administrator SERT 1 1 O client configurations 110 xml Selected Version 20150121 SUT Configuration Load Local File Download Latest With Browser Latest Web Version 20150108 SUT Discovery CPU Intel R Core TM i 2820QM CPU 2 30GHz 8 Total Threads akak Detected Test Environment PTDaemon Operating System bcinctod Microsoft Windows 7 Enterprise 6 1 7601 ected VM Configuration Windows Server 2012 JVM Review F cd Java Virtual Machine Criteria Matched Oracle Corporation 1 7 0 67 b01 Choose Test Suite HotSpot 1 7 0 u3 z D
12. Rating Tool SERT User Guide Controller System under Test runpower bat sh serthost bat sh SPEC PTDaemon Host JVM runtemp bat sh SPEC PTDaemon director bat sh config all xml Client JVM 1 Director JVM Client JVM 2 ws Reporter Reports Client JVM N Figure 2 SERT Startup Procedure 6 May 2015 37 of 40 Copyright 1988 2015 SPEC Server Efficiency Rating Tool SERT User Guide 6 2 Generate report files with the reporter scripts In some cases the user might want to create a new HTML CSV or txt report from an existing SERT xml output file for example after editing user modifiable fields configuration information in the TestEnvironment section In order to re generate an HTML CSV or txt report from the raw files the following comment need to be used reporter bat options for Windows based system reporter sh options for Linux UNIX based system Required option r SERT xml results file Example reporter bat a r results sert 0002 results xml Additional options a Generate all report formats HTML CSV and Text default generate only HTML output c Generate CSV report files o Output Output file for generated report default match the filename of the raw file with a txt csv or html extension to match the report type May not be used with
13. a test and to view and archive the results of those tests 2 1 Prerequisites for Using the GUI The instructions below assume that the GUI will be running on the Controller system and that the SUT is a separate host 2 1 1 On the SUT and Controller system e Ensure that the firewall is disabled see section 3 for platform specific instructions 2 1 2 On the SUT only e lfapplicable please follow the platform specific settings in section 3 e Set the JAVA variable in the serthost sh bat script to the java executable SERT should use For example o For serthost bat Set JAVA C Program Files Java jdk1 7 0 bin java exe o For serthost sh JAVA usr java jdk1 7 0 71 bin java e Run Install directory NSERT 1 1 0Nserthost bat orserthost sh 2 2 Invoking the GUI Open a cmd window and run one of the scripts SERT 1 1 0NSERTUINrun sertui bat or run sertui sh depending on your operating system cd Install directory NSERT 1 1 0NSERTUI run sertui bat The command script doesn t return and will leave the window open Note If power and temperature data is shown as zero or invalid after a test run please verify your connections and power cycle the Power Analyzer as the SERT cannot currently repair this misconfiguration 2 3 Interacting with the GUI 2 3 1 Start The Start panel of the GUI shows the sequence of steps necessary to run the SERT 6 May 2015 8 of 40 Copyright 1988 2015 SPEC Se
14. airports or coffee shops Windows Firewall state On Incoming connections Block all connections to apps that are not on the list of allowed apps Active public networks 77 Marriott CONFERENCE See also Notification state Notify me when Windows Firewall blocks a new app Action Center Network and Sharing Center Select Turn Windows Firewall on or off from the left hand side of the Firewall applet Customize Settings Jel Windows Customize Settings v Search Control Panel 2 Customize settings for each type of network You can modify the firewall settings for each type of network that you use For your security some settings are managed by your system administrator Domain network settings 7 Turn on Windows Firewall Block all incoming connections including those in the list of allowed apps V Notify me when Windows Firewall blocks a new app Q Turn off Windows Firewall not recommended Private network settings 7 Turn on Windows Firewall Block all incoming connections including those in the list of allowed apps V Notify me when Windows Firewall blocks a new app x Turn off Windows Firewall not recommended Public network settings 7 amp Turn on Windows Firewall Block all incoming connections including those in the list of allowed apps V Notify me when Windows Firewall blocks a new app o Turn off Windows Firewall not re
15. are expected in this case 8 SUT Minimum Memory Requirement For the CPU and Hybrid worklets Chauffeur allocates a maximum of 256 MB heap space per hardware thread and reduces the heap if there isn t sufficient physical memory Sufficient is defined using the following algorithm heap space per thread 64bit physicalMemoryGB 1 0 85 hardwareThreads heap space per thread 32bit physicalMemoryGB 0 875 0 85 hardwareThreads Example Configuration 8GB physical memory 32 hardware threads heap space per thread 64bit 8 1 0 85 32 0 186 GB 190 MB In order to ensure that 256 MB will be used per JVM we can solve the formula above for physicalMemoryGB and we get minimumMemory 64bit 1 0 25 hardwareThreads 0 85 minimumMemory 32bit 0 25 hardwareThreads 0 74375 Example Configuration 32 hardware threads minimumMemory 64bit 1 0 25 32 0 85 10 4 GB Per thread heap space sizes below 256MB may result in lower performance for each client The SERT harness only ensures that all clients will be using the same heap size to avoid performance differences between them The formulas given above don t pertain to the memory worklets These worklets will use all available physical memory after subtracting the memory used by the operating system and other run time software JVM etc 6 May 2015 39 of 40 Copyright 1988 2015 SPEC Server Efficiency Rating Tool SERT User Guide 9 K
16. be set to the name of the file which will hold the log output Log files may not be shared so the names given to each must be unique If extended logging of range settings warnings errors and other important events is desired set the EXTLOG variable to TRUE e Start all power analyzer PTDaemon s runpower bat runpower sh and temperature sensor PTDaemon runtemp bat runtemp sh e Edit the test environment xml file in the SERT directory describing the hardware and software details of the test setup The default values in this file need to be changed to match the real test environment o For each additional power analyzer Add the following section to the test environment xml in the MeasurementDevices section and change the default values to match the real test environment lt PowerAnalyzer gt lt PTDaemonHostname gt localhost lt PTDaemonHostname gt lt PTDaemonPort gt 8888 lt PTDaemonPort gt lt HardwareVendor gt _Energy Minder Inc lt HardwareVendor gt lt Model gt _EM1000 USB lt Model gt lt SerialNumber gt _ser001122 lt SerialNumber gt lt Connectivity gt _USB2 lt Connectivity gt lt InputConnection gt _Default lt InputConnection gt lt CalibrationInstitute gt _NIST lt CalibrationInstitute gt lt AccreditedBy gt _IQ2 Calibration Laboratory lt AccreditedBy gt lt CalibrationLabel gt _N 32768 lt CalibrationLabel gt DateOfCalibration 2010 01 01 DateOfCalibration lt SetupDescription gt _Unknown l
17. device must be configured as JBOD or single disk RAID O array e The content of the storage devices being tested will not be erased For meaningful results however it is strongly recommended to reformat the filesystem on the storage devices before the SERT storage worklets are started for the first time The test data files created during the first test run can be reused in subsequent runs 1 3 3 Memory Requirements The following table shows the minimum memory requirements by total system hardware threads Minimum System Memory to Run SERT 1 23 28GB 5 24 37 gt 12GB po m 38 51 216GB E 9 52 78 224GB lt m o 79 105 232GB a 106 132 gt 40 GB 5 S eed 133 159 gt 48 GB me 5 2 x 160 187 256GB E F g g 188 214 264GB 2 o a O wm 215 268 280GB o 2 e amp 269 323 296 GB 324 377 gt 112 GB a E 378 431 2128 GB 2 432 486 gt 144 GB Note Some systems may be able to run with less memory than indicated in this table but performance may be reduced For exact minimum memory sizes within each range please see chapter 8 Memory should be evenly distributed between processors for best performance e g sockets or physical packages In the case of a two socket processor system the memory should be equally distributed between the sockets You may encounter WARNING messages stating that the coefficient of variation between clients was greater tha
18. during which the measured data may not be accurate and because PTDaemon cannot determine the accuracy of the data while the device is in auto ranging mode The SERT allows you to configure separate ranges for each power analyzer and can adjust the ranges as needed for each measurement interval These settings appear in the 1isteners xml file Each PowerAnalyzerListener includes voltage range and current range parameters The default configuration includes multiple current range settings for different load levels but all are set to the same value none This indicates that no change will be made to the settings on the device during a test run The range settings may need to be adjusted when making changes to the system under test the worklets being tested or changes to the configuration of the operating system The validation performed at the end of the run will determine if power measurement accuracy thresholds are not being met If your result includes warnings regarding the power analyzer uncertainty adjusting the current ranges may resolve the issue 4 1 Automating Range Settings using the GUI The Choose Test Suite panel gives you the option to select a Range Discovery run rather than the standard SERT full run By selecting this option the tool will run an abbreviated subset of the SERT worklets which should be sufficient to create range settings that will provide valid measurements in a majority of cases The steps to complete
19. following items need to be considered for a 32 bit ARM platform The variable ARM JAVA FP ABl in serthost sh must be set appropriately for your Java version The currently supported HotSpot 7 JRE uses the soft float ABI so the variable should be set to armel in that case For other JVMs please consult the vendor s documentation Since the supported Linux versions use the ARM hard float ABI while HotSpot 7 uses the soft float ABI several non default libraries may need to be installed in Linux so that HotSpot will function properly This may be accomplished via the command sudo apt get install y libc6 armel libsfgccl For processors running at speeds less than 2GHz some of the worklets may require extra warmup time in order to reach steady state performance before the measured intervals begin Default warmup configurations on all worklets are a single 30 second warmup but up to ten 30 second warmup periods are allowed These values can be modified manually in config all xml their names are cpuWarmuplntervalCount 6 May 2015 29 of 40 Copyright 1988 2015 SPEC Server Efficiency Rating Tool SERT User Guide memcCapacityWarmuplIntervalCount storageWarmuplIntervalCount and hybridwWarmuplntervalCount 5 warmup periods should be sufficient for processors running at speeds over 1GHz 3 6 Oracle Solaris If the Host process is not run as root discovery data will be incomplete as some commands that the discovery process runs
20. is still using a default value which does not reflect the actual system being measured If a section is listed as Complete with a green background this indicates that all fields within this section have been updated either through discovery value population or manual editing Test Environment Navigator Menu Test environment information is required to document the hardware and software used Start in a SERT Test Suite Edit each section by double clicking on the row SUT Configuration e Load a previously saved environment configuration by clicking LOAD SUT Discovery Description Items Information identifying the testers and their location 7 Section Test Environment Test Information SUT Basic Information Basic information about the system and its devices co PTDaemon JVM Configuration JVM Review Choose Test Suite Launch Test Results Help SUT CPU Information Information about system CPU s SUT Memory Information Information about system memory SUT Memory DIMM Details Information about memory DIMMs SUT Power Information about system power SUT Power Supply Details Information about power supplies SUT Disk Drives Information on system disk drives SUT Network Interfaces Information on system network interfaces SUT Other Information on other hardware installed on the system Software Operating System Informatio
21. the a option p Direct XSL transform must be used with x default generate a report from the transformed output s Skip validation Reports will automatically be marked invalid t Plain ASCII text output default generate HTML output x XslFile Use a non default XSL file to generate the report Here are some examples of how to use the reporter scripts for regenerating default report files or for generating special output files e regenerate all HTML report files reporter bat r results sert 0000 results xml e regenerate all default report files reporter bat a r results sert 0000 results xml e generate special CSV report files reporter bat s c x results XSL File csv detail xsl o results detail csv r results sert 0000 results xml Additional XSL scripts for generating other CSV report files are included in the results XSL File folder 6 May 2015 38 of 40 Copyright 1988 2015 SPEC Server Efficiency Rating Tool SERT User Guide 7 Coefficient of Variation CV The coefficient of variation CV is defined as the ratio of the standard deviation c to the mean py C H It represents the extent of variability in relation to mean of the population The Host CV and Client CV are presented as percentage numbers in the SERT report 7 1 Host CV The Host CV is calculated from all individual node performance Scores for the whole set of nodes population is shown in the report for each interval The Host
22. 5 35 of 40 Copyright 1988 2015 SPEC Server Efficiency Rating Tool SERT User Guide Check client configurations 110 xml is up to date and download the latest version from http www spec org sert docs client configurations 110 xml if necessary and copy to base folder Compare the version YYYYMMDD string in the header section of the file with your local copy where the version string specifies the date this file was last changed in the format year month day e g 20130529 On the Controller running the Director edit the SERT configuration file config all xml in the SERT directory o Search for id clientConfig and replace the string defaultConfig with the configuration name from client configurations 110 xml matching your processor OS JVM memory per client combination Use the guidance on the Client Configuration page http www spec org sert SERT JVM Options html for the correct configuration name Chauffeur picks the correct JVM heap sizes parameters Xms and Xmx number of JVMs and Java parameters based on the specifications in client configurations 110 xml for the selected reference configuration The computed values must be used for a valid measurement o Search for diskioFilePaths in the definitions section at the beginning Edit the following lines lt parameter id diskioFilePaths gt lt path gt specify pathname of storage test device here lt path gt lt parameter gt Change remo
23. CVs are calculated and displayed for multi node runs only for single node runs a 0 0 is displayed The Host CV is used by SERT to identify significant deviance of results of some nodes from the mean Similar results per node are expected for a homogeneous set of nodes i e the CV should be fairly small High CV values indicate greatly different results for one or more nodes CV values above certain thresholds will cause warnings or error messages in the SERT result files and may lead to a run being invalid and thus unsuitable for submission 7 2 Client CV The Client CV is calculated from all individual client performance Scores for the whole set of clients JVM instances including all nodes for multi node runs population is shown in the SERT report for each interval The CV is used by SERT to identify significant deviance of results of some clients from the mean Typically similar results per client are expected i e the CV should be fairly small High CV values indicate greatly different results for one or more clients For unbalanced configurations e g different types of storage devices HDD and SSD or dissimilar amounts of memory attached to different processors high CV values are normal CV values above certain thresholds will cause warnings or error messages in the SERT result files and may also lead to a run being invalid and thus unsuitable for submission Warnings can be ignored for unbalanced configurations as high CV values
24. Efficiency Rating Tool SERT User Guide Save In C SERT 1 1 0 Y a amp c B amp 8 C3 checkers C3 redistributable sources 3 clie discovery results D clie J Documentation C SERTUI 3 clie images tools 3 clie Ci lib 3 client o out 3 ciie CI PTDaemon 3 client 1 out 3 ciie 4 Ji I FileName Files of Type All Files ly The code will automatically pick the correct JVM options for each workload from the specified set and start the client JVMs with these options The option sets for all supported platform OS JVM combinations have been defined by the platform manufacturer and must be used for compliant SERT runs 2 3 7 JVM Review The JVM Configuration Review panel allows the user to review the JVM options and the number of client JVMs used for each workload during the measurement HEI seRT 1 1 0 Navigator Menu Start SUT Configuration SUT Discovery Test Environment PTDaemon JVM Configuration JVM Review Choose Test Suite Launch Test Results Help JVM Configuration Review Please review the JVM options and configuration Current client configuration ID CPU Storage Hybrid Memory Idle Intel Win HS17 1n Number of Clients Option Set Configuration H7n JVM Options XX UseParallelOldGC XX AggressiveOpts XX UseLargePages XX ParallelGCThreads 2
25. Manufacturer Standard disk drives a e 2 7 Tum off Windows write cache buffer flushing on the device To prevent data loss do not select this check box unless the device has a separate power supply that allows the device to Device status flush its buffer in case of power failure This device is working properly More information about write caching settings This right shows write caching enabled click on the checkbox to disable it e g the tick will disappear 6 May 2015 26 of 40 Copyright O 1988 2015 SPEC Server Efficiency Rating Tool SERT User Guide 3 1 2 Disabling the Windows firewall The firewall may be disabled by the following Open the Windows Control Panel and then click on the Windows Firewall applet e Windows Firewall T MP Control Panel All Control Panel Items Windows Firewall v Gror Help protect your PC with Windows Firewall Windows Firewall can help prevent hackers or malicious software from gaining access to your PC through the Allow an app or feature Internet or a network through Windows Firewall Mj Change notification settings For your security some settings are managed by your system administrator Turn Windows Firewall on or i off m 7 Domain networks Not connected Qj Restore defaults Advanced settings m 7 Private networks Not connected v ee m 7 Guest or public networks Connected Networks in public places such as
26. Standard Performance Evaluation Corporation SPEC Server Efficiency Rating Tool SERT User Guide 1 1 0 7001 Heritage Village Plaza Suite 225 Gainesville VA 20155 USA SPECpower Committee Server Efficiency Rating Tool SERT User Guide 1 2 3 Table of Contents Overvie W oorner eaaa a a R a a a a e E 4 a e SUMMMANV A EE E E E O T EE T EEE 4 1 2 Setup Requirements iisscccicciscccsiccsbesdciscesetsccseestecdeceeseccsdencedsseeeseccacesdeadacesssacecencedecesebsadeeesteaces 4 1 3 SUT Configuration Requirements ceeee eese eese eene eene eene enn nonne na totes e sa soe ses astee sena rose sonans 5 1 3 1 Platform Req irementsa c oet o xe ere ires co code d bo eee nee She dee sdenecs o Cea eed een d ea Ree aed RT UP rin 5 1 3 2 Storage Requirements RECEPIT a ei lerr aE ESE a Ei lar EE ESSERE OSa Eai 5 1 3 3 Memory Req irements eerie ct essevesnaaseveseesseveshaavacectonseUashaatesecnebss echansesechensbuechanscsecmebsstacnaatese 6 1 3 4 Network REGUIFEMENTS 25 c o omes qe etti qu esee o ts cos eee due xi ue ee nee EA TA pae ee idee A eR 6 1 3 5 Software Requirements ese ert reete Ree ER HR ere EYE Cer AS UP RECEN EIN ERN OP ERE ES PER ISn eere Ege E aee Rede re cree veda de 7 1 4 Ed aWnhcltpnmeE TT E 7 Starting SERT via the Graphical User Interface GUI esee eee eee eene 8 2 1 Prerequisites for Using the GUI
27. all enabled by default For information on configuring or disabling IPSec see the AIX Knowledge Center https www 01 ibm com support knowledgecenter ssw aix 71 com ibm aix security ipsec htm 3 4 IBM PowerLinux On IBM Power systems running RHEL 6 x or SLES 11 x with more than 32GB of memory large pages must be configured manually Sign on as the root user Download the newest libhugetlbfs library from the project home at the following URL http sourceforge net projects libhugetlbfs files Execute the command cat proc meminfo grep MemTotal This returns the total memory available to the operating system in Kilobytes Value Compute the target large pages as follows targetLP Truncate Value 18204 Note 18204 is a factor designed to allocate approximately 90 of available memory for large pages Execute the following command to set the number of large pages echo targetLP gt proc sys vm nr hugepages Create the necessary pseudo filesystem by executing the following commands mkdir libhugetlbfs groupadd libhuge chgrp libhuge libhugetlbfs chmod 770 libhugetlbfs usermod root G libhuge this assumes root is the SERT user on the system mount t hugetlbfs hugetlbfs libhugetlbfs Edit etc fstab and add the following line hugetlbfs libhugetlbfs hugetlbfs mode 0770 gid 1000 0 0 Execute the following command to mount the huge table space hugeadm create global mounts 3 5 32 bit ARM Platform The
28. bling the storage write cache on Microsoft Windows ccccccccccecssssssscecececeesesesseceesceeseeeaeeeesceeees 26 3 1 2 Disabling the Windows firewall esses enne nnne enne nn tnn neni sn en neta nasse 27 3 2 Red Hat Enterprise Linux and SUSE Linux Enterprise Server eeeeeeee eee ee enne 28 3 2 1 Disabling the storage write cache on LINUX ecececeessscecececeeseeececececseseaeeecececeeseeaeseseesceesesaeaeeeeseeeees 28 3 2 2 Disabling the RHEL6 x SLES firewall cccccccsssccsscccsssceescecsecesseecsssceeseecasceceeecssseeeaeecssseeeaeecesseeeneeseas 28 3 2 3 Disabling the RHEL Z firewalls uon Eee et erae ee aa eee nen to dera e EEUU ne ee dene RAN 28 e e P e e 28 3 3 1 Disabling the ADCfirewall etr rcr e ete bete aet rn adv E ESSERE 29 3 4 IBM PowWerLin x erre eter E prete d etna co EN po oo oar EAEE oe Fan aao eo anke id peu ea ale nn Ep oe po 29 3 5 32 bit ARM Platform eiie center ee 16e erae cce eckeloncccssecutadveccetestaccesssvedsoussesecusadseccetas 29 LH PNEES HC ECGIETICS 30 3 6 1 Disabling the storage write cache on Oracle Solaris seeesesesesess eee enne 30 6 May 2015 2 of 40 Copyright 1988 2015 SPEC Server Efficiency Rating Tool SERT User Guide 3 6 2 Disabling the Solaris firewall ee duret o eene ene a eee ue to aeo Need re E eee e ede YE RNE 30 3
29. ce Range Settings panel see picture below fa PTDaemon Device Range Settings Range Settings of PTDaemon Power Analyzer Below settings will be written to listeners xml Make necessary changes as needed parameter xmins http spec org power chauffeur name voltage range gt lt range set gt lt default range gt 230 lt default range gt lt range set gt lt parameter gt gt parameter xmins http spec org power_chauffeur name current range gt lt range set gt lt workloads gt Memory lt workloads gt lt worklets gt Flood lt worklets gt lt default range gt 1 lt default range gt lt range set gt lt range set gt lt workloads gt Memory lt workloads gt lt worklets gt Capacity lt worklets gt lt default range gt 1 lt default range gt lt range set gt lt range set gt lt workloads gt Storage lt workloads gt lt default range gt 0 5 lt default range gt lt range set gt lt range set gt lt workloads gt idle lt workloads gt lt default range gt 0 5 lt default range gt lt range set gt lt _ Default ranges gt lt range set gt lt default range gt 1 lt default range gt sranae level 87 5 gt 1 lt ranae gt Il The default ranges none shown in the example above must be replaced by a suitable current or voltage value Range definitions for other workloads worklets and or intervals can be added here as well
30. commended Click on the Turn off Windows Firewall for the relevant network s e g Domain Private or Public then click on OK 6 May 2015 27 of 40 Copyright 1988 2015 SPEC Server Efficiency Rating Tool SERT User Guide 3 2 Red Hat Enterprise Linux and SUSE Linux Enterprise Server Transparent Huge Pages THP must be enabled when running the SERT in a 64 bit Linux environment Earlier versions of RHEL and SLES do not enable this feature by default although other untested distributions may do so To determine whether Transparent Large Pages are enabled for the system run the following command cat sys kernel mm redhat transparent hugepage enabled RHEL Or cat sys kernel mm transparent hugepage enabled SLES11 SP2 The user should see the following output always never with the always in brackets indicating that Transparent Large Pages are enabled by default on the system If this file exists but the output shows never or mmio please run the following command at startup to enable Transparent Large Pages echo always sys kernel mm redhat transparent hugepage enabled for RHEL Or echo always gt sys kernel mm transparent hugepage enabled SLES11 SP2 3 2 1 Disabling the storage write cache on Linux For instructions on how to disable the different write caches please read the manuals provided by the manufacturers of the system controller and storage devices Different versions of Linux p
31. done as follows e Onthe Start menu click Run In the Open box type gpedit msc The Group Policy dialog box opens The user account running this tool needs to be a system administrator e Onthe Group Policy console expand Computer Configuration and then expand Windows Settings e Expand Security Settings and then expand Local Policies e Select the User Rights Assignment folder The policies will be displayed in the right hand details pane e Inthe pane double click Lock pages in memory e Inthe Local Security Policy Setting dialog box click Add e In the Select Users or Groups dialog box add the user account that will be used to run the SERT typically Administrator e Restart the machine for these changes to take effect 3 1 1 Disabling the storage write cache on Microsoft Windows For systems with Windows Server installed each storage device will usually appear in the Device Manager To disable write caching select the desired storage device in the Device Manager then either double click or right click and select Properties Select the Policies tab to display the interface that allows the disabling of write caching a R LIC Mp UIN NN Policies Volumes Driver Details ST3250310AS Improves system performance by enabling write caching on the Device type Disk drives device but a power outage or equipment failure might result in data
32. e valid measurement results for a full run you may need to use one of the manual processes described below 4 2 Manual Range Settings Optional To configure the ranges manually replace each of the voltage range and lt current range gt entries in the listeners xml file with a value appropriate for your system under test The range setting should generally be slightly higher than the maximum expected reading during that interval Each model of power analyzer will support a specific set of ranges however the configuration can use any value and PTDaemon wvill round the range settings up to the next supported range 4 3 Extracting Range Settings from Full Runs Optional The SERT also includes a process for extracting appropriate ranges for your configuration after a full run has been completed After a run completes the results files will be stored in a numbered directory in the results directory for example results sert 0007 results xml Use the directory name from that test and run the following command reporter bat s p r results sert 0007 results xml x org spec power sert resources rangeSettings xsl o rangeSettings txt or reporter sh s p r results sert 0007 results xml x org spec power sert resources rangeSettings xsl o rangeSettings txt This will generate a rangeSettings txt in the same directory as results xml This file will contain an XML listener specification for each power analyzer used in the
33. ed to the controller and specify the interface address 6 May 2015 14 of 40 Copyright O 1988 2015 SPEC Server Efficiency Rating Tool SERT User Guide e Calibration Specify the date of last calibration for the power analyzer e Logging If needed save a log and or debug output file by clicking on the checkbox This should not normally be used except under guidance from SERT support personnel After making your choices click on Test Connection to start the connection between the power analyzer and the controller Be patient as this process may take 30 seconds or more Power for the different SERT worklets and load levels may vary significantly This may result in current readings which fall outside the limits of the given ampere range This will lead to invalid power readings or high uncertainty values 2196 in both cases the result will be marked invalid In order to avoid such conditions power ranges must be adjusted dynamically during a SERT test run Also the power analyzer voltage range must be set to match the standard voltage of the test environment Proper setting of the voltage range is a prerequisite for correct calculation of uncertainties Both voltage and current ranges can be specified individually per workload worklet and load level in the listeners xml file or under View Edit Range Settings in the PTDaemon configuration panel Range settings from previous runs will be imported from 1isteners xml by the SERT UI a
34. em must be in a state sufficient to execute a class of server applications larger than the tool alone Disabling operating system services is forbidden 1 4 SERT Installation Ensure that a Java runtime environment is already installed on the SUT and the Controller Start the SERT setup via java jar SERT 1 1 0 setup jar and follow instructions on the screen Ensure that SERT is installed on the OS partition for both SUT and Controller Installing the SERT distribution media will create a directory tree of files and subdirectories L SERT 1 1 0 checkers discovery Documentation images lib PTDaemon redistributable sources results L XSL File SERTUI archive config images lib logs tools This document can be found in the Documentation directory while the various script and batch files discussed in the following sections are present in the SERT directory or the PTDaemon sub directory If the user will be using multiple machines to perform a SERT test sequence it will be necessary to unpack or install all the files onto the storage devices which hold the operating system image on each system 6 May 2015 7 of 40 Copyright 1988 2015 SPEC Server Efficiency Rating Tool SERT User Guide 2 Starting SERT via the Graphical User Interface GUI The SERT GUI provides an intuitive way for the user to configure and run
35. erC 2 CpuNumberOfChips 1 v MemorySizeMB 32719 v KeyboardName Enhanced 101 or 102 key v MouseName USB Input Device v Monitor Yes v OSVendor Microsoft Corporation Y OSName Microsoft Windows Server 2012 R2 Standard W OSVersion 6 3 9600 v OSFileSystem NTFS v BiosVersion V4 6 5 4 R2 10 0 for D3239 A1x v NicDescription Intel R Ethernet Connection I217 LM NicDescription Intel R Ethernet Connection I217 LM v NicDescription Intel R 1210 Gigabit Network Connection Ex lar cam P J Discovery items are not being saved Click NEXT to continue is a Copyright 1989 2015 Standard Perfomance Evaluation Corporation SPEC All rights reserved 6 May 2015 11 of 40 Copyright 1988 2015 SPEC Server Efficiency Rating Tool SERT User Guide 2 3 4 Test Environment The Test Environment panel allows the user to provide detailed information about the hardware and software setup used to obtain the SERT results The information as defined in the Test Environment fields will appear in the Results Report at the end of the test run Initially the default template is populated with sample fields to provide an example configuration The discovery process will also populate some fields in certain sections of the test environment file If the status of any specific section is listed as Defaults in use displayed with a yellow background this means that at least one of the fields listed within this section
36. etecten Launch Test Matched Configuration Results Client Configuration ID Intel_Win_H817_1n Reset Use Custom ID Help Workload Client Count Option Set CPU jg H n Storage Determined at test launch H2 Hybrid 8 H n Memory 8 H8n Idle 1 H1 Successful Configuration Match Click Next to use this configuration 2 Back Next Exit Copyright 1989 2015 Standard Perfomance Evaluation Comoration SPEC All rights reserved Note Before selecting configuration parameters make sure that the local copy of the client configurations 110 xml file matches the official version of this file which is available on SPEC s web site at http www spec org sert docs client configurations 110 xml The GUI will check this for you automatically it is reported as the Latest Web Version If necessary you can set a web proxy in the GUI startup scripts run sertui bat or run sertui sh If the version strings don t match you can download the latest version by clicking on the Download Latest With Browser hyperlink After you have downloaded the latest version click on the Load Local File button for opening this version in SERTUI see picture below In case the controller doesn t have internet access you have to find another system with internet access for copying the file on a removable storage medium and then transferring it to a folder on the controller system 6 May 2015 19 of 40 Copyright 1988 2015 SPEC Server
37. fficiency Rating Tool SERT User Guide Note According to the SERT run rules write caches must be disabled for all storage devices You must select the Write cache disabled check box to confirm the storage write caches is disabled If this check box isn t selected a warning will be displayed later when the test is launched Click Next to continue after making the selections Note If you start SERT and the specified storage path name does not exist the following message will appear Storage verification failed on SUT lt SUTNAME gt Storage path lt PATHNAME gt does not exist Would you like to continue launch of test suite Note If there is insufficient free space on the specified storage device s the following message will appear Storage verification failed on SUT lt SUTNAME gt not enough free space available Would you like to continue launch of test suite If either error appears it is highly recommended that the test be terminated immediately and the reported error be corrected before restarting the SERT If the test is not terminated immediately it will eventually fail 2 3 9 Launch Test The Launch Test panel is used to initiate a SERT test and to monitor its progress Click Launch to begin the test Cancel is available and may be used as necessary inix Launch Test Navigator Menu Launch the chosen Test Suite by clicking on the LAUNCH button Progress and status Start information will appear below durin
38. g the test Launch Test SUT Configuration HP Number of Runs 1 Launch Cancel SUT Discovery m Progress Test Environment Current Phase 0 eee Worklet 0 JVM Configuration Test Suite 0 Estimated Time Remaining JVM Review Status Chauffeur Choose Test Suite Launch Test Results Help Ready to launch test Back Next Exit Copyright 1989 2015 Standard Perfomance Evaluation Comoration SPEC All rights reserved 6 May 2015 22 of 40 Copyright 1988 2015 SPEC Server Efficiency Rating Tool SERT User Guide After a few seconds Status messages and Progress information will be displayed and the test will proceed as intended Estimated Time Remaining is approximate and might be significantly longer for high end configurations especially multi node configurations In some cases the Progress Bar and Estimated Time Remaining may appear to not show any progress even though the SERT test is functioning correctly BRI SERT 1 1 0 s inixi Launch Test Navigator Menu Test Config Selected Full Test Suite config all xml Start SUT Model SPEC System EM65 CPU Name Tacoma Spi Memory 2560 Full Discovery Data Launch Test SUT Configuration Number of Runs 1 Launch Cancel SUT Discovery Progress Test Environment Current Phase 0 PTDaemon Worklet 0 JVM Config
39. he results can be browsed at any time by clicking on View The test is now complete Exit the GUI by clicking on Exit 6 May 2015 24 of 40 Copyright 1988 2015 SPEC Server Efficiency Rating Tool SERT User Guide 2 3 11 Help The Help panel provides release information and links to various documents made available to the user Help Navigator Menu Start U ser Information and Documentation SUT Configuration SERT 1 1 0 SERTUI 1 1 0 20150303 SUT Discovery Test Environment PTDaemon JVM Configuration DOCUMENT DESCRIPTION JVM Review Design Document Describes SERT design principles and details including the underlying workload Choose Test Suite Run and Reporting Rules Describes the procedures needed to produce measured valid SERT results Launch Test Measurement Setup Guide Describes PTDaemon configuration including details for specific hardware units Results User Guide Describes SERT test configuration including workload definition and guidelines for running a test Help Credits Credits and acknowledgments Back Next Exit Copyright 1989 2015 Standard Perfomance Evaluation Corporation SPEC All rights reserved 6 May 2015 25 of 40 Copyright 1988 2015 SPEC Server Efficiency Rating Tool SERT User Guide 3 Platform Specific Settings 3 1 Microsoft Windows For a 64 bit environment it is necessary to enable the Lock Pages in Memory user rights policy which may be
40. hen no results have been created or when some other part of the test pass has failed then you may see one or more up to five instances of the following message The system cannot find the file specified Once the script completes navigate to the SERTlog subdirectory and copy the latest archive file zip for Windows tar gz for Linux to the system for submission to SPEC for analysis The files will be created in the following format hostname YYYY MM DD HHMM zip tar gz where YYYY is the year MM is the month DD is the day and HHMM represents the hours and minutes count Always ensure the latest log archive file is captured as a new version will be generated in the SERTlog subdirectory every time the collectlogfiles script is run on that system Follow the above procedure for the controller system ensuring the correct script is executed for the Operating System running on the controller Use the form on http www spec org sert feedback issuereport html in order to submit the files Note Technical support requiring involvement of specialists is done by volunteers from our member institutions http www spec org spec membership html Please do not send us any proprietary information in your queries If you feel you cannot clearly ask your question without reference to proprietary information please clearly mark such communications and we will not involve our SERT specialists but rather will answer your query only if SPEC s
41. i e He Doederl Eie Os RR ed essa da V HM REA RA FMA S AAOS TERES 40 SVN Revision 1920 SVN Date 2015 05 11 14 25 03 6 May 2015 3 of 40 Copyright 1988 2015 SPEC Server Efficiency Rating Tool SERT User Guide 1 Overview 1 1 Summary Various government agencies around the world are currently working on Energy Efficiency Regulatory Programs for servers The SPECpower Committee designed implemented and delivered the Server Efficiency Rating Tool SERT a next generation tool set for measuring and evaluating the energy efficiency of computer servers This User Guide is intended for both novice and experienced SERT testers It is structured with the basics to get a novice user up and running quickly and includes a considerable amount of additional details for more experienced testers The SERT is composed of several elements including e The test harness called Chauffeur handles the logistical side of measuring and recording power data along with controlling the software installed on the SUT and controller system itself e The Director instructs the System under Test SUT to execute the workload o TheSUT can be a single or multi node server e The SERT Workload Suite exercises the SUT while the test harness collects the power and temperature data e The SPEC Power and Temperature Daemon PTDaemon connects to the power analyzer and temperature sensor and gathers their readings while the workload executes e The Repo
42. n 5 if the memory is not evenly distributed 1 3 4 Network Requirements At least one Network Interface Controller NIC with a minimal transfer rate of 1 Gbit s 6 May 2015 6 of 40 Copyright 1988 2015 SPEC Server Efficiency Rating Tool SERT User Guide 1 3 5 Software Requirements Required software components per server host are e Exactly one operating system instance with one user space including all modules that are installed during the default installation process The OS must be from the list of supported software platforms http www spec org sert docs SERT JVM Options 110 html e Ifthe system uses a hypervisor by default the SERT may run in a guest instance virtual machine of that hypervisor Only one guest may run on each host The guest must be configured to use all hardware resources that are not reserved by the hypervisor Full support for virtualized environments is considered for the future e AJavarun time environment including one or more instances of a Java Virtual Machine JVM o A 64 bit JVM is required on 64 bit platforms and a 32 bit JVM is required on 32 bit platforms o Aversion of a Java run time environment from the list of supported software platforms e For the purposes of running the SERT please ensure that the user is logged on as an Administrator on Windows or as root on other platforms Optional power management software when installed must be reported The operating syst
43. n about the OS installed on the system Software Other Information about other software installed on the system JVM Information about the JVM and its configuration Firmware Information about system BIOS and management firmw Notes Additional notes on system details and configuration amp j NjtNvjojmiojo sejolj c mej co Click NEXT to continue when editing is complete Copyright 1989 2015 Standard Perfomance Evaluation Corporation SPEC All rights reserved The SUT Discovery from the previous step only partially identifies the configuration Every section needs to be checked by the user and entries must be edited manually as appropriate These elements must be double checked for correctness and examined for proper formatting Highlight a row in the table and double click to edit the information associated with that section e g select the Test Information section To make changes double click the field to be edited in the right column change the value directly and hit the Return key On completion click OK to save 6 May 2015 12 of 40 Copyright 1988 2015 SPEC Server Efficiency Rating Tool SERT User Guide Besides changing the available entries some test environment sections support adding new groups of items Currently this feature is available for Memory DIMM Details and for Disk Drives Click on the Add Grou
44. nd only need to be edited after configuration changes It is strongly recommended though to check the settings for correctness before starting a test run Starting with SERT version 1 1 0 the GUI offers a method to partially automate the specification of range values by running a brief test prior to a full SERT run See Section 4 for instructions on performing a range setting run serrano im PTDaemon Configuration Navigator Menu Power Analyzer 1 Temperature Sensor1 Start K e Enabled __ View Edit Device Description ViewiEdit Range Settings SUT Configuration ries SUT Discovery Measurement Device Test Eswiroamont UNSUPPORTED DEVICE Il be marked inv PTDaemon Interface JVM Configuration C JVM Review Calibration Choose Test Suite 7 sr Launch Test Logging v Results Help Q STARTED Stop o Click NEXT to continue when configuration is complete Back Next Exit Copyright 1989 2015 Standard Performance Evaluation Corporation SPEC All rights reserved 6 May 2015 15 of 40 Copyright O 1988 2015 SPEC Server Efficiency Rating Tool SERT User Guide For the Temperature Sensor tab there are three sections e Measurement Device Choose which temperature sensor is being used from the drop down menu e Interface Choose how the temperature sensor is connected to the controller and specify the interface addres
45. nge set gt lt workloads gt Storage lt workloads gt lt default range gt none lt default range gt lt range set gt lt range set gt lt workloads gt Idle lt workloads gt hf hf lt default range gt none lt default range gt lt range set gt lt default range gt none lt default range gt range level 87 5 gt none lt range gt range level 75 gt none lt range gt lt range level 62 5 gt none lt range gt range level 50 gt none lt range gt lt range level 37 5 gt none lt range gt range level 25 gt none lt range gt lt range level 12 5 gt none lt range gt lt range set gt lt parameter gt lt parameters gt lt listener gt e Onthe system under test edit the SERT host script file serthost bat serthost sh both be found in the SERT directory for the appropriate system configuration For single runs the KEEPALIVE variable can be cleared for multiple consecutive runs it should remain at the default setting keepalive Check the following specifications for proper values matching the test system configuration JAVA for a non default Java Virtual Machine JVM change the java command to the pathname of the java command you want to run The serthost sh bat script files currently both include the default definition JAVA java On the system under test call the serthost bat sh script file 6 May 201
46. nown Issues Pease check for the latest updates http www spec org sert docs SERT FAQ html 10 Trademark SPEC and the names SERT SPECpower ssj and SPEC PTDaemon are trademarks of the Standard Performance Evaluation Corporation Additional product and service names mentioned herein may be the trademarks of their respective owners 11 Copyright Notice Copyright 1988 2015 Standard Performance Evaluation Corporation SPEC All rights reserved 6 May 2015 40 of 40 Copyright 1988 2015 SPEC
47. p Item button to add a new group of items The example in the picture below shows three new detail items for DIMM group 1 The information in the value fields must be edited as described above Use the Delete button to delete the highlighted items Use the OK button to store the updated information and return to the main Test Environment panel i o S Test Environment Section Editor b o Section SUT Memorv DIMM Details Item value H Number of Dimms 0 4 i Size of Dimm MB 0 8192 i Description of Dimm 0 8GB 2Rx8 PC3 14900R ECC CL13 slots 1 3 5 and7 populated zm EZ Number of Dimms 1 4 size of Dimm MB 1 4096 Description of Dimm 1 4GB 2Rx8 PC3 14900R ECC CL13 slots 2 4 6 and 8 populated Add Group Item Delete OK Cancel A Note that all sections of the test environment information must be complete before continuing further This is indicated in the right column of the table showing a green color for all sections By clicking on the Save Copy button the user can save a fully populated template for subsequent re use Use the Load button to re load an existing template in place of the current default template After editing test environment fields you can preview the System Under Test section of the result report file by clicking on the Preview Report button
48. range setting run Open the file in a text editor and follow the instructions which will be to copy some text from that file to the 1isteners xml 5 Using the Collector Scripts The SERT package is distributed with two log collection scripts to assist in packaging up various log files and other critical configuration files for troubleshooting The collector scripts are intended to be run on both the SUT as well as the controller system The two files are Operating System specific and may be found in the base SERT installation directory collectlogfiles bat is intended to be executed on a Microsoft Windows Operating System collectlogfiles shisintended to be executed on a Linux or AIX system In order to help SPEC diagnose a failed run please follow these steps 1 Ensurethe run has completed and the GUI has been stopped on the Controller side 2 Onthe SUT open up a command line window change directories to the base SERT installation directory and execute either collectlogfiles bat for Windows systems or collectlogfiles sh for Linux systems If you required to share your results files please use the r option e g collectlogfiles bat r in order to package up the latest results files various log files and other critical configuration 6 May 2015 32 of 40 Copyright 1988 2015 SPEC Server Efficiency Rating Tool SERT User Guide 5 If either of these script files are executed when the SERT has not yet been executed w
49. require elevated privileges The psrset 1M command is used to effect process affinity and that requires either root privileges or the PRIV SYS RES CONFIG privilege See privileges 5 and usermod 1M for instructions on assigning that privilege to a normal user The Solaris virtual memory system will not overcommit memory so it is necessary to configure more swap space than is default in order for the SERT Memory workload to run properly This swap space will not actually be used and will therefore not adversely affect scores it is only necessary for very large memory allocations to succeed Large memory systems may require swap space equal to the size of main memory though on small memory systems it may be sufficient to configure swap space equal to about half the size of main memory Disks used exclusively for swap space must also be included in the configuration for the Storage workload It is recommended that those disks be partitioned with a small 20GiB UFS partition for SERT use with the rest of the disk dedicated to swap The SERT Storage workload does not support measuring any kind of RAID or other disk aggregation This includes ZFS in any multi disk configuration All ZFS pools on the system must consist of only a single disk ZFS filesystems used for SERT Storage workload data set must have the primarycache and secondarycache properties set to either none or metadata The easiest way to accomplish this is to create a separate filesys
50. rovide different mechanisms for disabling disk caches so it may be necessary to contact the manufacturer of your storage device s if neither the manufacturer s manuals nor the operating system documentation explain how to do this 3 2 2 Disabling the RHEL6 x SLES firewall The firewall may be disabled by issuing the following command as root chkconfig iptables off A reboot is required for the change to take effect 3 2 3 Disabling the RHEL7 firewall For RHEL7 and other Linux distributions that use systemctl the firewall may be disabled by issuing the following command as root systemctl disable firewalld A reboot is required for the change to take effect 3 3 IBM AIX It is necessary to setup large pages correctly Sign on as the root user Execute the command bootinfo r This returns the total memory available to the operating system in kilobytes Value Compute the target large pages as follows targetLP Truncate Value 18780 Note 18780 is a factor designed to allocate approximately 87 2596 of available memory for large pages Execute the following commands vmo r o lgpg regions targetLP o lgpg size 16777216 o v pinshm 1 bosboot aD shutdown rF This will cause the system to be rebooted with the large pages set up correctly for the SERT 6 May 2015 28 of 40 Copyright 1988 2015 SPEC Server Efficiency Rating Tool SERT User Guide 3 3 1 Disabling the AIX firewall AIX does not have a firew
51. rter gathers the environmental power and performance data after a run is complete and compiles it into an easy to read format e A Graphical User Interface GUI to simplify setting up and executing the SERT Additional background information describing the design and implementation of the SERT may be found in the SERT Design Document http www spec org sert docs designdocument pdf 1 2 Setup Requirements For the most basic SERT hardware measurement setup one of each of the following is required e System under Test SUT the actual system for which the measurements are being taken e Controller e g a server desktop PC or laptop the system to which the power analyzer temperature sensor and the SUT are connected o The Controller and SUT are connected to each other via an Ethernet connection o The Power Analyzer and Temperature Sensor are connected to the Controller via device specific means o Hardware requirements o A hardware configuration capable of running a supported OS and JVM combination o Atleast one Network Interface Controller NIC with a minimal transfer rate of 1Gbit s O At least two communication interfaces for controlling a power analyzer and a temperature sensor o Software requirements o OS 32 or 64 bit x86 Current versions of Microsoft Windows OS and most current Linux distributions o Acurrent Java 7 or 8 run time environment o Vendor specific drivers supporting special power analyzer and
52. rver Efficiency Rating Tool SERT User Guide BHI SERT 1 1 0 loj xl Start Navigator Menu re Welcome to the Server Efficiency Rating Tool SERT 1 1 0 SUT Discovery Test Environment To run a test follow the Wizard through the sequence below PTDaemon 1 SUT Configuration Specify the SUT and verify connectivity JVM Configuration 2 SUT Discovery Run an automated tool to inventory SUT hardware and software configuration JVM Review 3 Test Environment Provide a complete set of information about your test environment Choose Test Suite 4 PTDaemon Configure Power and Temperature Daemons 5 JVM Configuration Identify and choose the set of JVM options best suited for your system Launch Test 6 JVM Review Review and verify the JVM options Results 7 Choose Test Suite Choose the Test Suite you would like to run and view its configuration id 8 Launch Test Launch the Test Suite and observe its progress through different phases elp 9 Results Examine the results and save them to a location of your choice Click on the NEXT button below to begin Copyright 1989 2015 Standard Perfomance Evaluation Corporation SPEC All rights reserved Click Next to continue or Exit to terminate the GUI 6 May 2015 9 of 40 Copyright 1988 2015 SPEC Server Efficiency Rating Tool SERT User Guide 2 3 2 SUT Configuration The SUT Configuration panel allows the user to configure the IP Addres
53. s e Logging If needed a log and or debug output file can be saved by clicking on the checkbox After making your choices click on Test Connection to start the connection between the temperature sensor and the controller 2 3 5 1 View Edit Device Description The following figure shows an example of a power analyzer device description Similar description panel is available for temperature sensor devices Description Item SerialNumber CalibrationLabel ser001122 N 32768 Model __EM1000 USB SetupDescription Unknown AccreditedBy IQ2 Calibration Laboratory Calibrationinstitute _NIST InputConnection _Default Connectivity Identifier _USB2 _EM1000 HardwareVendor Energy Minder Inc Information shown here is imported from the test environment xml file Each field can be edited individually to match the given configuration Specifically you need to ensure that the power analyzer has been calibrated within one year before running the test The corresponding CalibrationLabel must be specified in the appropriate field The device description is copied to the test environment xml file for use in subsequent SERT runs 6 May 2015 16 of 40 Copyright O 1988 2015 SPEC Server Efficiency Rating Tool SERT User Guide 2 3 5 2 View Edit Range Settings Voltage and current ranges can be specified in the PTDaemon Devi
54. s es of the SUT ini xi SUT Configuration Navigator Menu Enter information about your System Under Test SUT below Click VERIFY to let SERT validate the information given Start SUT Info SUT Configuration Network Address Local SUT SUT Discovery IP Address Test Environment Verify SUT PTDaemon Node IP Address Result JVM Configuration JVM Review Choose Test Suite veny Launch Test View Discovery Data Results Help o Click Verify to perform SUT verification and discovery Copyright 1989 2015 Standard Perfomance Evaluation Corporation SPEC All rights reserved Specify whether the IP Address of the SUT is the Local Host or provide the IP Address es Multiple IP Addresses can be provided via a comma separated list e g 10 0 0 101 10 0 0 110 or by specifying an IP Address range e g 10 0 0 101 120 Click Verify in order for the GUI to verify the connection to the remote service running on the SUT s If a failure occurs an error message will appear in the lower left corner of the GUI When verification is successful click Next to continue 6 May 2015 10 of 40 Copyright O 1988 2015 SPEC Server Efficiency Rating Tool SERT User Guide 2 3 3 SUT Discovery The SUT Discovery panel shows software and hardware configuration retrieved from the SUT Please wait for the process to complete after which a table of discovery items will be shown
55. such a run are as follows 1 Onceatthe Choose Test Suite panel select the Range Discovery choice from the dropdown menu 2 For most power analyzers it is safe to check the Use Auto Ranges box and let the analyzer select the range choices for the run 3 Complete the remaining settings on this page as documented previously in the User Guide Proceed to the launch page and launch the run 5 When the run completes you will see a message in the lower portion of the panel stating that range settings have been updated 6 Return to the PTDaemon configuration panel stop the PTDaemon instance and then review the range settings 7 Return to the Choose Test Suite panel select Full Test Suite and continue on to make a full run gt 6 May 2015 31 of 40 Copyright 1988 2015 SPEC Server Efficiency Rating Tool SERT User Guide Some power analyzers may be unable to select appropriate ranges automatically and the range setting run will result in invalid values for ranges In such a case you may need to edit range settings from the PTDaemon panel setting all range values to none and then manually set the analyzer to a range that will measure the system at its highest load and rerun the Range Discovery test with Use Auto Ranges disabled The Range Discovery run should give working settings in most cases but it is impossible to cover all system types and analyzer types If range values generated by the run do not giv
56. t SetupDescription gt lt PowerAnalyzer gt o For multi node environments the node quantity needs to be set An example for a 3 node environment Node lt Quantity gt 3 lt Quantity gt 6 May 2015 34 of 40 Copyright 1988 2015 SPEC Server Efficiency Rating Tool SERT User Guide e Ifthe default port numbers in the PTDaemon script files have been changed the 1isteners xml file in the SERT directory must be changed accordingly Also any range settings for the power analyzer can be specified in the 1isteners xml configuration file o For each additional power analyzer Add the following section to the 1isteners xml and change the default values to match the real test environment Note Do not forget to change the port number listener enabled true gt lt type gt PowerAnalyzerListener lt type gt lt classpath gt lt path gt lib ptdaemonClientApi jar lt path gt lt classpath gt lt parameters gt lt parameter name hostname gt localhost lt parameter gt lt parameter name port gt 8888 lt parameter gt lt parameter name current range lt range set gt lt workloads gt Memory lt workloads gt lt worklets gt Flood lt worklets gt lt default range gt none lt default range gt lt range set gt lt range set gt lt workloads gt Memory lt workloads gt lt worklets gt Capacity lt worklets gt lt default range gt none lt default range gt lt range set gt lt ra
57. tem for the SERT Storage workload data set with a command such as zfs create o mountpoint sert data0 o primarycache metadata o secondarycache metadata rpool sert data When possible UFS should be preferred as the filesystem to use for the SERT Storage workload It provides significantly better performance than ZFS in no cache conditions 3 6 1 Disabling the storage write cache on Oracle Solaris The cache on individual storage devices may be disabled using the expert mode of the format 1M command This command requires root privileges to run For each disk attached to the system run format e path to disk Enter the cache command then the read cache subcommand and then select disable Repeat for the write cache subcommand There is a shell script in the tools subdirectory called solaris disable disk caches sh which will perform these actions automatically for all disks attached to the system It also reports whether the cache was successfully disabled for each disk Many Solaris based systems ship with hardware RAID controllers with on board cache If it is possible to disable that cache it will be done with an HBA specific configuration utility such as MegaCli for LSI based RAID controllers Consult the documentation for your specific server and HBA model to determine how the cache can be disabled If it is not possible to disable the cache on the hardware RAID controller it may not be used for a compliant SERT run
58. uration Navigator Menu Power Analyzer 1 Temperature Sensor 1 Start v Device Enabled View Edit Device Description View Edit Range Settings Listening Port 8888 SUT Configuration Remote PTDaemon IP Address 192 168 0 1 Measurement Device SUT Discovery Dummy testing only v Test Environment UNSUPPORTED DEVICE Results will be marked invalid Analyzer Channel Number PTDaemon Interface JVM Configuration amp Serial GPIB USB Ethernet JVM Review Interface Address com1 Calibration Choose Test Suite C Specify calibration date Date of last calibration yyyy mm dd 2014 09 03 Launch Test Logging C Save log output to a file outlog Results C Save debug output to a file debug log Help Q A I Debug log verbosity 0 Off 1 minimum 9 maximum B8 7 2 32 4 5 8 FP 8 Not Started Test Connection Add Remove ri Click NEXT to continue when configuration is complete Copyright 1989 2015 Standard Performance Evaluation Corporation SPEC All rights reserved For the Power Analyzer tab there are four sections e Measurement Device Choose which power analyzer is being used from the drop down menu Once a device has been selected only the options appropriate for that device will be displayed in the remaining sections e Interface Choose how the power analyzer is connect
59. uration Test Suite ex Estimated Time Remaining JVM Review Chauffeur Power 8888 Temperature 8889 Choose Test Suite Launch Test Results Help v Ready to launch test MEE a Back Next Exit Copyright 1989 2015 Standard Perfomance Evaluation Corporation SPEC All rights reserved When the test completes click Next to continue 6 May 2015 23 of 40 Copyright 1988 2015 SPEC Server Efficiency Rating Tool SERT User Guide 2 3 10 Results The Results panel allows the user to view and archive the results from the test pass that has just completed Results Navigator Menu View and Archive Test Results Start Test Suite Results Test Folder Time Completed SUT Configuration sert 0000 03 03 2015 18 25 42 SUT Discovery Test Environment Save As PTDaemon Arcane JVM Configuration JVM Review Choose Test Suite Archived Results Test Name Description Launch Test Results Help e e Copyright 1989 2015 Standard Perfomance Evaluation Corporation SPEC All rights reserved Click View to load the results in the default Web browser Click Save As to save the results folder to a location of choice Click Save to name annotate and save the results folder to the default Archive directory and enter them into the table provided Once placed in the archive t
60. ve add path names of the storage devices to be tested Each path name should be listed in its own lt path gt tag Edit the director bat sh script files for single runs or the start director multiple bat sh script files to start the desired number of SERT runs First the host IP address must be specified in director bat sh then edit the following line HOSTS hostname or IP of Host systems _ o For multi SUT environments Add each SUT IP address separated by a comma to the HOST argument Note Do not use a space behind the comma s An example for 3 SUTS HOSTS 10 10 10 1 10 10 10 2 10 10 10 3 On the controller system call the director bat sh script file for a single run or the start director multiple bat sh script file for multiple consecutive runs The duration of the whole test sequence depends on the number of test cases defined above The result files for each SERT test run results html and results xml are stored in a separate directory with a name beginning with sert nnnne g sert 0000 On current releases of the SERT a typical test pass exercising all worklets CPU memory and Storage I O on a typical two socket server with 16GB or more of memory and at least two on board storage devices may take four hours or more This duration is governed by the amount of hardware present in the SUT so more richly populated platforms may take longer to execute 6 May 2015 36 of 40 Copyright 1988 2015 SPEC Server Efficiency
61. vi Choose Test Suite Random v Hybrid v Launch Test Ss A Memory vj m Results Storage Path Setup v Write cache disabled for all disk drives Path Help specify pathname of storage test device here Delete Storage Path Add Storage Path Click NEXT to continue after choosing a Test Suite 2 Back Next Exit Copyright 1989 2015 Standard Perfomance Evaluation Corporation SPEC All rights reserved In the Storage Path Setup section specify the paths of all configured storage devices under test Separate pathname fields are required for each storage device More devices can be added using the Add Storage Path button Each time the Add Storage Path button is clicked a new pathname field is created at the bottom of the existing list of pathnames as shown in the following figure Clicking on the Delete Storage Path button will delete the currently selected pathname field You must enter a pathname referencing a filesystem on a storage device here The following requirements apply to pathnames e The system partition e g C under Windows must not be specified here Instead only the pathname for the additional test partition must be given here e t is recommended to use descriptive pathnames for mounting the test storage devices and to avoid drive letters for pathname specification Example SAS 10K O0 instead of K under Windows OS 6 May 2015 21 of 40 Copyright O 1988 2015 SPEC Server E

Server Efficiency Rating Tool (SERT) User Guide

Contents

Download Pdf Manuals

Related Search

Related Contents