Home

D7.2.3 Cloud services modeling and performance

1. V 90 dlc E 96 ee ca toD ANE A M Lo nca NE 101 Appendix B Availability ale EE 105 Appendix C Performance Profile sssccscssssessonssecconssscnecsussectoussscnenssseseouesseonssscnenssvestonssss 112 Appendix De Cost NONE faassen Shes pam opted tae tenant nance a dece Mte DR MR E MM PM DERI ERE 117 Appendix E Security ele EE 123 Appendix F Provider model example cccccccssseccccessccccesecceceeececeeseceeeuececsueceesensecessuneees 124 Project Title ARTIST Contract No FP7 317859 www artist project eu Page 7 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework Table of Figures FIGURE 1 OVERALL CLOUD MODEL GENERATION PROCESS eee e hme nhe 19 FIGURE 2 OVERALL METHODOLOGY FOR EVALUATING NON FUNCTIONAL GOALS ccr 22 FIGURE 3 HIERARCHICAL VIEW OF CLOUDML ARTIST sseeeesseeeeeemmm eene 25 FIGURE 4 CORE GA EEN 26 FIGURE 5 SPECIFIC PROVIDERS PROEILES I eee nennen hnnne nennen nnn een nns nnne nnn 26 FIGURE 6 SUPPORTING PROFILES errrrrnnnrrnnnrrnnnnrnnnnrnnnnennnnsvnnnnrnnnnvnnnnennnnnvnnnsvnnnnsnnnusnnnnsnnnnnvennusennusnennssene 24 FIGURE 7 OVERALL BENCHMARKING SUITE ARCHITECTURE ee eene enne 37 FIGURE 8 SCENARIO FOR THE ETICS AND ARTIST REPOSITORIES ee 39 FIGURE 9
2. CloudSleuthGenericAvailability Real 1 AverageProviderDefinedAvailability Real 1 Figure 47 Availability measurement stereotypes AvailabilityElement parameters are shown in the following table Name Type Ca Gard Description M NN Gi ico NN of tsType service offering ProviderSpecificAvail ProviderSpecificAvail Provider s specific ability abilityStatsType availability based on each provider s definition GenericAvailabilityStatsType subtype expresses availability statistics that are measured by CloudSleuth me mm qmm TimePeriodInDays Integer 1 Time period of Statistics collection Project Title ARTIST Contract No FP7317859 www artist project eu Page 105 of 129 D7 2 2 Cloud services modelling and performance Version v1 0 Final Date 15 09 2014 analysis framework chosen through official site of Cloudsleuth Generic availability vailability based on Cloudsleuth equation 1 In the following table the properties contained in the ProviderSpecificAvailabilityStatsType stereotype are described The concrete value for this field can be acquired through the use of the Abstracted Availability Auditor Library 3ALib that we have developed Further information regarding this library can be found in the content of D7 1 EE FailedPercentageOfSL he percentage of As ins due to an unavailable VM AverageProviderDefin The average of edAvailability availability for each provider
3. memory Integer 1 platform OSs 1 processorArchitecture Architecture Types 1 Stereotype laaSlnstanceType instanceCategory InstanceCategories DH region AvailabilityZones L operatingSystem 05s 1 virtualCores Integer 1 memory Real 1 localDisk Integer 1 processorArchitecture Architecture Types 1 networkPertormance NetworkPerformanceTypes DH autoScaling Boolean 1 Project Title ARTIST Stereotypes TaaSNetworkService max bandwidth in Integer 0 1 Stereotype laasProvider Instancelypes laaSInstanceType 1 max bandwidth out Integer 0 1 Stereotype aasstoragesService type IaaSStorageTypes 1 capacity GB Real 1 region AvailabilityZones 1 replication type Storage ccountReplicatinTypes Stereotype aasFeatures virtual private cloud offering Boolean 1 hypervisor Hypervisors 1 Version v0 6 Final Date 31 03 2015 laasDatal ypes Enumeration Hypervisors Enumeration aaSStorageTypes Raw storage Volume storage Object storage Block blobs Page blobs Tables Queues Figure 44 laaS Stereotypes Contract No FP7 317859 Page 91 of 129 Enumeration OSs CentOS RedHatEnterpriseLinux Debian OracleEnterpriseLinux WindowsServer SUSELinuxeEnterprise Ubuntu FreeBSD openSUSELinux SUSEEnterpriseLinux Enumeration Architecture Types Enumeration InstanceCateg
4. BenchmarkResult workload WorkloadType 1 Figure 50 Illustration of the basic part of performance model and the OCL constraint Next is described the property contained in BenchmarkResult stereotype Project Title ARTIST Contract No FP7317859 www artist project eu Page 112 of 129 D7 2 2 Cloud services modelling and performance Version v1 0 Final Date 15 09 2014 analysis framework bo c NN Me EIER workload patterns that can be mapped to concrete applications DacapoResult properties are described in the following table The DaCapo benchmarks reflect performance time and are used in order to evaluate Java based applications Cardinality in all results can be also 0 in case no tests have been performed for this specific benchmark PerformanceTime d Response time for test completion With regard to YCSBResult stereotype YCSB reports back a number of metrics such as runtime throughput number of operations average minimum and maximum latency These are in cluded in the performance profile in order to describe overall results of an offering Name Wwe Card Description Gc MENU ull d Response time for test completion throughput S Execution time needed for workload completion averageLatency 0 1 Update operations completed minLatency 0 1 Average time per operations the Client measures the end to end latency of executing a particular operation against the database Projec
5. The repository will contains binary packages of benchmarking tools that can be downloaded and installed on the machine where the benchmark is executed Each package is identified by four coordinates e package name the name of the tool e g YCSB Dwarfs Filebench e version the version of the tool Multiple versions of a given tool may exist in the repository mainly for compatibility reasons e platform the platform on which the package is meant to be installed e g CentOS Debian Windows e format the format of the package Depending on the platform the installation package might be in different formats e g tar gz zip rom deb msi For a given tool binary packages stored in the repository can be obtained either by compiling the source code or by taking original distribution packages or by repackaging original distribution packages A possible layout for a http front end for the repository that makes it possible to download benchmarking tools from the ARTIST repository is the following repository_host package_name version platform package_name lt tar gz rom deb ms 1 Moreover in case of platforms that allows it a view of the repository in a platform dependant software repository might be provided For instance an access to the repository in the APT format used by Debian platforms for installing software and or YUM format used by RedHat platforms might be provided In order to simplify and partially
6. 0 to 66 _id status backendStatus loginPort publicAddresses privateAddress imageld hardware os id type 537cbc3d0d64 AVAILABLE stopped 22 Array 0 Array 0 us east l ami 10 Keys gt 5 Keys m us east 1 i 6a NODE 537cbc3d0d64 AVAILABLE stopped 22 Array 0 Array 0 us east l ami 10 Keys 5 Keys m us east 1 i 9a NODE 53985e458e7fc UNAVAILABLE stopped 22 Array 0 Array 0 us east 1 ami 10 Keys sp 5 Keys amp us east 1 i ba NODE 53985e458e7fc UNAVAILABLE stopped 22 Array 0 Array 0 us east T ami 10 Keys sp 5 Keys m us east 1 i 9a NODE 53986a34cdb7 UNAVAILABLE stopped 22 Array 0 Array 0 us east 1 ami 10 Keys m 5 Keys e us east 1 i 6a NODE 53986a34cdb7 UNAVAILABLE stopped 22 Array 0 Array 0 us east l ami 10 Keys sp 5 Keys m us east 1 i 9a NODE 53986af8eefec UNAVAILABLE stopped 22 Array 0 Array 0 us east 1 ami 10 Keys sp 5 Keys amp us east 1 i ba NODE 53986af8eefec UNAVAILABLE stopped 22 Array 0 Array 0 us east 1 ami 10 Keys sp 5 Keys m us east 1 i 9a NODE 53986bc52994c UNAVAILABLE stopped 22 Array 0 Array 0 us east l ami 10 Keys m 5 Keys us east 1 i 5a NODE 53986bc52994c UNAVAILABLE stopped 22 Array 0 Array 0 us east l ami 10 Keys sp 5 Keys m us east 1 i 9a NODE 53986c8a0eefc UNAVAILABLE stopped 22 Array 0 Array 0 us east l ami 10 Keys sp 5 Keys amp us east 1 i ba NODE 53986c8a0eefc UNAVAILA
7. 1 homepage String 1 P plafformOffering 0 1 NetworkOffering PlafformOfferin sla Real 0 1 saaSOffefing 0 1 SaaSOfferina sla Real 0 1 ProcessingOfferind sla Real 0 1 serviceDescription String 0 1 serviceType ServiceTypes scalabilityType ScalabilityType 0 1 performanceSLA PerformanceSLAMetric 0 1 availabilitySLA Integer 0 1 service service serviceName String 1 Version v0 6 Final Date 31 03 2015 Stereotype MeasurementType test value Real 0 1 periodicityOfTest Real 0 1 average Real 0 1 iterations Integer 0 1 standardDeviation Real 0 1 coefficientOfVariation Real 0 1 higherlsBetter Boolean 0 1 Figure 42 CloudMI Artist s high level stereotypes Stereotype PerformanceSLAMetric typeOfOperation String 1 guaranteedValue Real 1 higherlsBetter Boolean 1 metricUnit String 1 referencelnterval Real 0 1 intervalUnit String 0 1 acceptedVariation Real 0 1 Stereotype MetricAndValue metric String 1 value String 1 Enumeration AvailabilityZones East_US North_Central_US South_Central_US West_US East_Asia Southeast Asia Brazil_South North Europe West Europe Japan East Japan West Stereotype ScalabilityType scalabilityCapabilities ScalabilityCapabilitiesType 1 scala
8. Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework etc with the main objective of having a high level design of the application via Reverse Engineering techniques and a static analysis of the source code These data in combination with the ideal technological maturity identified in the maturity assessment as well as with the target platform requirements will provide some metrics such as how much effort will be needed to perform this migration and how complex it will be 3 Business Feasibility Analysis Based on the technical feasibility and the results from the ideal situation identified in the maturity assessment and the identification of the characteristics that the target platform shall have a business feasibility analysis is performed This business feasibility analysis is aimed to provide not only economic information such as ROI or payback but also which are the main risks to be faced with the migration and the organizational processes affected by the uptake of the new business model Two of this steps the Technical Feasibility Analysis and the Business Feasibility Analysis the information related to the target description will be used as input to the process In the Technical Feasibility Analysis target platform requirements are needed in order to propose the correct migration task for each component propose changes in the component model and estimate the complexity required to ca
9. INSERT AverageLatency ua 1123 8473 INSERITI MinLarency usm 435 INSERT MaxLatency us 1005352 rustet S thPercentileLatencyima 4 INSERE S99thPercentileLatencyims 4 INSERT Feturnet 100000 INSERT 0 05454 INSERT 1 4493 IBSERI INSERT IBSERT INSERT EIESERT INSERI I LI Ws J ou a b ul RA RJ d J d J Qu LI Hs Le e FJ ga cl Ri de C deb ge CN p INSERT 8B 1 INSERT 9 BLIBSERI 19 3 INSERI ii 1 e L4 Ifi FE Line 25 1013 Column 12 Character 51 33 Moche Figure 20 YCSB output example 4 2 4 2 Dwarf Berkeley dwarfs is a benchmarking suite aiming to cover a variety of elementary operations that correspond to higher level applications In ARTIST we will be utilizing an extended version created in the context of the BONFIRE project 8 This version enables the remote installation of the executables plus a number of additions in the result reporting e g incremental load adjustment The tests involve the following applications areas e Dense and sparse matrix operations e Spectral analysis e MapReduce e Structured grids e And N body methods The configuration can be performed via a control file in which various parameters such as test sizes incremental workloads and iterations can be set Results can be directly downloaded from the target environment through a provided script Result reporting include various St
10. ol RR ERN T Tn 15 1 1 ADOUt this deliverd e EE 16 1 2 OC UI al S UC E 16 1 3 MENN EE 16 1 3 1 CIO UCI WE EE ae E 16 1 3 2 Pei 18 da Sv U UNE T T 17 1 3 3 2j w m 17 1 4 Fitting into overall ARTIST architecture esssessesssseesesrresssrrererrrrrssreresrrrresrrerererereeresene 18 1 4 1 Management of Cloud models and benchmarking results 19 1 4 2 Exploitation of cloud models in modernization assessment process 19 1 4 3 Usage of cloud models in the forward engineering process 21 1 4 4 Usage of cloud models in product testing and validation 22 1 4 5 Integration with the other Artist repositor 22 2 CIoudMLQOARTIST Implementation eese nnne nnne 24 2 1 F lhctonal SS gedet EE 24 2 2 FINN 25 2 2 1 VE ele E WE ed LE 25 2 2 2 UML Profiles Reide TION WEN 27 2 2 3 FN 28 3 CloudML ARTIST Delivery and Usage 29 3 1 PENN 29 3 2 Eee dead ae dvs 0 EEE NN 29 3 3 Licensing WTO RE ACO EEE EE 35 3 4 PO VU GE E 35 4 Benchmarking Tools Implementation sees 36 4 1 Functional Description cceeccccssecceesececceeccceeeseccesausececsuneceeseeseceseuensesenecetsenecetsenes 36 Project Title ARTIST Contract No FP7 317859 www artist project eu Page 5 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 ana
11. 03 2015 analysis framework e anindicator related to the Service Efficiency Metric it could be more than one in case of more performance data e achart describing the performance trend for each test it could be more than one in case of more performance data Obviously the views will be different for each tool depending on own database structure and performance metrics to measure Java Application Reports x localhost v C Q Search wh A 4 e SG fal Pid visitativ SSHtoAmazonEC2 M ovirt Engine User Por s DMSF ARTIST R amp D AREAPERSONALE Adempiere python 2 7 libcloud E Cloud Bench Panel Dashboard Settings Scientific Graphics Map Reduce DaCapo Results Chart 120 000 4 Data Streami ag 29498 14 38 100 000 Media Streaming 80 000 4 60 000 Software Testing gt 40 000 Web Search 20 000 4 em ge HE a a Web Serving at at a gt ah Af ai e e e xe e 409 e Filesytem Category Java Applications YCSB Chart data Test Number Cloud Provider Platform PerformanceTime ms Date Testi flexiant redhat 115523 2014 09 18 02 26 39 Test2 flexiant redhat 19773 2014 10 14 21 27 42 Test3 flexiant redhat 18002 2015 02 05 11 28 15 Test4 flexiant redhat 18760 2015 02 12 00 58 38 Figure 15 GUI controller integration So far it is not possible to link directly the GUI with the Controller in order to carry out the test directly by the interface however the component re
12. A GRE Kaka l Jl SCT T 3 OG 548 n9 120 05 8 BB B B 3 E S8 B vd ia rs TIE Tao Helg I k men Figure 27 FINCOS output example Actually the execution is allowed through an interactive GUI this means that the usage does not fit totally with the benchmarking controller functionalities cause it is not possible to use the own API to configure the several scenarios So far we could use the controller to install the CEP engine remotely and retrieve the results of the performance test locally Unfortunately the Project Title ARTIST Contract No FP7 317859 www artist project eu Page 52 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework configuration steps remains manual This issues will be solved in the future since a new version of the framework including any ready to run benchmark should be released Currently we are evaluating a draft solution proposed by FINCOS itself that would allow the extension of the framework in order to run tests in a not interactive mode 4 2 5 Benchmarking experiments and related performance model population In order to estimate the performance of the cloud resources for a successful migration the usage of the benchmarking tools described in the previous sections is needed The measurement results from the benchmarking process are incorporated in the CloudML ARTIST framew
13. Date 15 09 2014 analysis framework BackOfflnterval Identifying failure with a predefined level of delay between the requests In order to simplify the modelling framework of the Availability SLA definition we have defined different enumeration fields that include information for supporting the three aforementioned fields namely PaaSProviderAbstractedDefinition laaSProviderAbstractedDefinition and Stor ageProviderAbstractedDefinition in which SLAAbstractedDefinition is specialized Apart from enumeration fields Figure 49 also DiscountType stereotype is illustrated In this field compen sation can be linked in order to describe what is promised as compensation by cloud providers in the case of failures Enumeration Enumeration Enumeration StateDiscovery Stereotype PaaSOptions BackOffIntervalDefinition Status DiscountType HighReplicationDatastoreOption GAE 7 Connection MinPercentage Real 1 Multi amp ZOption AmazonRDS E Ping MaxPercentage Real 1 TotalCallsIncludedInUptime Azure SQL zur BoundedResponseTime Discount Real 1 Figure 49 Illustration of discount type and different enumeration fields that complete the SLA defini tion model The properties contained in the PaaSOptions enumeration are described in the following table Each of these options is unique and is related to a specific cloud provider me mm HighReplicationDatas high replication toreOption GAE datastore option enabled used by G
14. Profile PaaS InstancesService type InstanceTypes 1 name String 1 memoryOffered String 1 cpuOffered String 1 instancesService PaaSDataStorageOffering datastoreAccessType DatastoreAccessType sqlStorageSe Stereotype SqalStorageService dataStructureType DataStructureTypes 1 nonSqlStorageService 5 Stereotype NonSalStorageService type NoSQLTypes 1 Stereotype PaaSSoftwareOffering applicationCoatainerService 1 Stereotype ApplicationContainerService name ApplicationContainers 1 name Runtimes 1 programmingLanguaje ProgrammingLanguages 1 applicationFramework ApplicationFrameworks 1 Figure 45 PaaS Stereotypes Project Title ARTIST Version v1 0 Final Date 15 09 2014 PaaSDataT ypes Spring Framework 3 2 5 Spring NET 200 Spring Framework3 1 Rails Sinatra Play 2 0 Lift Django 0 96 Django 1 0 2 Django 1 1 Struts2 Enumeration NoSQLTypes KeyValue Column Document Contract No FP7317859 Page 98 of 129 JavaDataObjects JavaPersistenceAPI Blob Enumeration NoSqlProducts GoogleBigTable AmazonSimple ApacheCouch AppEngineDatastore GoogleCloudStorage Enumeration DataStructureT ypes Table Queue Enumeration ApplicationContainers Orade Weblogic Application Server Apache Tomcat JBoss Application Server
15. The most relevant innovative elements are related to the capacity offered by CloudML Artist to model the SLA offered by providers in terms of availability and performance Furthermore the metamodel defines the necessary structure in order to directly insert performance information of Cloud offerings in a variety of application types through the use of relevant benchmarks This structure includes also the workload under which the test was performed In addition placeholders have been included in order to keep historical information on availability Statistics of the Cloud offerings either in the form of generic availability measurements e g performed through the CloudSleuth web site or adapted to each provider s definition which is an indication of how frequently SLAs are violated by each respective provider for the specific offering 1 3 2 Benchmarking Suite The most innovative aspect of the Benchmarking Suite is in the management of the benchmarking process In fact the Benchmarking Suite is a fully automated solution that through connectors is able to provision cloud resources install and execute benchmarking tools retrieve results and eventually destroy provisioned resources Compared to other available benchmarking solutions for Cloud the Artist Benchmarking Suite drastically reduces the manual intervention in the benchmarking process allowing massive and large scale tests Such suite allows carrying out tests and getting perfo
16. The type of binding either how the communication happens with the service specification i 1 A pointer towards a file containing the actual service description A short name for identifying the service http en wikipedia org wiki RSDL http en wikipedia org wiki Web Services Description Language Project Title ARTIST Contract No FP7317859 www artist project eu Page 103 of 129 D7 2 2 Cloud services modelling and performance Version v1 0 Final Date 15 09 2014 analysis framework language The language in which the service is being used important for the use of software libraries A Configuration description needs to be specified for all services and libraries used Every service however requires different configuration properties so they preferably need to be generic The typical properties are e Application User Id e Secret Key The table below gives a short overview of the different properties belonging to the Property stereotype Name e om mm section String 0 1 A category of the properties in order to further group the different properties key String 1 The key identifying the property value String 1 The value belonging to the particular key Project Title ARTIST Contract No FP7317859 www artist project eu Page 104 of 129 D7 2 2 Cloud services modelling and performance Version v1 0 Final Date 15 09 2014 analysis framework Appendix B Availability p
17. addressed using a freely assignable logical URI independent of the organization of the repository This logical URI has to be unique in the repository A logical URI pattern for the benchmarking results of the previous example could be http www artist project eu benchmark result lt provider gt lt tool gt lt timestamp gt In addition to the actual document and additional meta data can be stored alongside the artefact The ARTIST repository can be accessed from an Eclipse workspace via a repository client plug in Most of the functionality of the repository will also be available via a web service interface 1 4 2 Exploitation of cloud models in modernization assessment process The modernization assessment process consists of three main steps 1 Maturity assessment The objective of this step is to analyse how mature the application is in terms of technology and business and how the customer wants the application to be in those two axes once the application is migrated The evaluation of the current situation and the ideal situation allows ARTIST to perform a gap analysis described in terms of a technical feasibility analysis and the business feasibility analysis 2 Technical Feasibility Analysis This analysis aims to provide a snapshot of how well bad designed an application is how complex and tightly coupled it is how interrelated Project Title ARTIST Contract No FP7 317859 www artist project eu Page 19 of 129 D7 2 3
18. almost 4000 seconds observed at the maximum value Before each VM is pinged the general connectivity is tested via 4 packets towards 8 8 8 8 Google DNS with an average time of 45 milliseconds per packet resulting in 0 36 seconds of overhead for this step Probability 0 500 1000 1500 2000 2500 3000 3500 4000 Sample Interarrival Delay Seconds Figure 34 Sample interarrival time probability distribution Table 4 Sample Statistics for the Amazon SLA experiment US East Overall Samples US East 89892 Maximum Sample Delay 3965 Average Sample Delay 40 065 seconds Project Title ARTIST Contract No FP7 317859 www artist project eu Page 64 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework Standard Deviation 21 488 seconds One interesting aspect for future work would be the comparison of the Cloud providers API responses to get the server list since this seems to be the bottleneck of the overall process This action needs to be performed before each sample in order to ensure that no intermediate user action such as shutting down one or more VMs has resulted in a service setup that is not eligible for an SLA Better response times would result in the ability to sample with higher frequency than the one observed in this case average of almost 40 seconds per sample 6 2 5 SLA metrics 6 2 5 1 SLA Adherence Levels Given the differences in the pro
19. appears in the following table InstanceNumber Integer 1 The number of VM instances MultipleAZ Boolean 1 The need for different Availability Zones The need of restart template before validating an unavailability sample RestartFromTemplate Regarding the storage layer the properties that have been identifying after examining the three SLAs and have been included in StorageProviderAbstractedDefinition are ErrorRateLimit PerCent and BackOffinterval The first one is related to the metric Error Rate which is defined as follows Failed Sam plesInInternal Error Rate TotalSam piesInInternal While the definition for the three providers is the same some details may differ such as the duration of the interval Amazon 5 minutes Google 10 minutes and Azure 1 hour and the fact that Azure defines a failed sample not only in terms of success or failure but also in terms of the delay of the response The second one BackOfflnternal is necessary only for Azure and Google and is defined as the defined as the requirement not to enquire the service status con stantly in case of failure but with a predefined level of delay between the requests different in definition for each provider Wee ee om mem ErrorRateLimitPerCen Integer 1 Percentage of Error Rate metric Project Title ARTIST Contract No FP7317859 www artist project eu Page 108 of 129 D7 2 2 Cloud services modelling and performance Version v1 0 Final
20. at any time based on how the connectivity validation is decided The possibility to create an abstracted method based on Jclouds in order to check via ssh the connection ability will be investigated however there may be some implications for the resources that have been created without the usage of Jclouds Thus we followed the generic approach of pinging However in order for this to apply one must have properly configured the VM instances to allow ICMP traffic so that the ping messages can reach the resources Special attention must be given also to the way the results are retrieved in the calculation of availability Small differences in implementation may result in significant delays in the query processing For example cursor count should be stored in a local variable rather than computed in each for loop since in the MongoDB driver case it is not included in any metadata but is calculated each time from the resultset 6 2 3 Technical Specification 3ALib is a java based library Java 7 It also has a strong dependency from Apache Jclouds li brary 1 7 for abstracting from the Cloud providers API and uses MongoDB 2 4 4 as a backend for storing the analytical logs for availability It also has dependencies from external jars Indica Project Title ARTIST Contract No FP7 317859 www artist project eu Page 62 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework t
21. authentication methods that can be specified through a list of elements of type AuthenticationMethod Currently OAuth SAML HMAC and Basic are provided as the ones most used nowadays However many more can be easily added in the future An additional category field is added which is related to the functionality provided by the SaaS service Here the same categorization of services is used as described in D7 3 defined by an enumeration type SoftwareServiceCategory These categories are identity multi tenancy security monitoring logging billing caching and messaging Every service exposes itself through a number of Interfaces These interfaces are of a certain type named BindingType For web services most of them are either HTTP REST or SOAP services In order to support client libraries as a kind of API the binding type can be specified as being a Language In this case the language needs to be specified as well through the corresponding property The actual API client library or web service description will be captured through the specification property which points to a URI There are numerous interface description languages already available e RSDL e WSDL These specifications are eventually can be used to generate stub code glue code to either use and or access the API The following table gives an overview of the different properties belonging to the Interface stereotype es Jm Je Iren bindingType ServiceBinding
22. downloading the third party tools is the following link http etics res eng it repository pm registered repomd name artist benchmarking index html Project Title ARTIST Contract No FP7 317859 www artist project eu Page 58 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework 6 3ALib Implementation 6 1 Functional Description The 3ALib Java library is an abstracted way of taking measurements regarding the availability of services regardless of the supported provider on which they are deployed Furthermore the library in its final version will be able to calculate the actual levels of availability as defined in the respective providers SLAs initially laaS Compute level SLAs are supported based on the latter s definition of the term for more details on the differences between providers consult D7 1 12 An example of an laaS level SLA is Amazon EC2 Compute SLA http aws amazon com ec2 sla What is more the library may check for which running in stances of the user the SLA applies based on the preconditions and necessary deployment options that are considered obligatory by the providers in order for an SLA to apply So overall the goals of the library are e inform users on why their services do not adhere to the SLA e keep detailed logs of the availability of services e calculate availability levels based on the providers definition and be used for compen
23. e g cost of the specific offering Project Title ARTIST Contract No FP7 317859 www artist project eu Page 24 of 129 D7 2 3 Cloud services modelling and performance Version vO 6 Final Date 31 03 2015 analysis framework CloudProvider Soctal Netwnrke Storage GR D men Service Bus Deployment Network Billing Pricing bility Security Benchmarks Monitoring Multitypes these types may be assigned to various aa5 levels Consist of two type groups a generic features one declaration per provider e eg billing granularity b service offering features different for each offering e g cost Defined separately as packages and referenced where needed inthe higher level Figure 3 Hierarchical view of CloudML ARTIST For easier management these types will be defined in separate sub packages and will be referenced in the basic part of the meta model 2 2 Technical Description CloudML G ARTIST has been created as a set of UML profiles able to cover the modelling needs both at WP7 and WP9 level At WP7 level ClooudML ARTIST should be focused at creating profiles for each provider under study in order to select the optimal one for migration depending on the characteristics of the legacy application At WP9 level CloudML ARTIST CAML should allow to enrich the PIM Platform Independent Model generated by reverse engineering techniques WP8 As a result a Platform Specific Model PSM will be generated in order to al
24. ese senes 50 FIGURE 26 FINCOS FRAMEWORK ARCHITECTURE eee I RI eene hene n eser nennen renun 5 FIGURE 27 FINCOS OUTPUTENAMPLE I IIR HneR hn e nnne nene nnne sn sen se sen see e enses nene nan 32 FIGURE 28 PERFORMANCE TIME IN MS FOR DACAPO WORKLOADS e ene enne 54 FIGURE 29 BENCHMARK PROFILE IS APPLIED TO THE DIAGRAM eene mene 54 FIGURE 30 BENCHMARK PROFILE IS APPLIED TO THE AMAZON EC2 PROFILE enne SE FIGURE 31 INSTALLATION PROCEDURE OF BENCHMARKING TOOLS eee 57 FIGURE 32 SALIB GENERAL RTE 60 FIGURE 33 3ALIB CLASS DIAGRAM eeeeeeee IR nennen nn nen nse nein sese shines sene ese se eese ense nun 61 FIGURE 34 SAMPLE INTERARRIVAL TIME PROBABILITY DISTRIBUTION eee 64 FIGURE 35 3ALIB BACK END DB VIEW essernrrnnnnrrnnnrnnnnrrnnnrrnnnnrnnnnennnnennnnvnnnnnnnnnennnnsnnnnnvnnnnnnnnnnsnnnnsnennsseen 69 FIGURE 36 3ALIB CONFIGURATION FILE ccccccccccceecccescccesecceuecscescccuueceecseeccscenecsueceeeceseeeeseeeseueeeeaes 69 FIGURE 37 3ALIB AVAILABILITY CALCULATION INTERVAL DEFIDNITION eR Ta FIGURE 38 MODEL UppnATERPROCESS III He hene hene nnnnn hene hse e ees h eee ente ense enne nna T3 FIGURE 39 ABSTRACT CLASS BENCHMARKRtSUUTs eee eee mene nen nnns en nnne nnne 74 FIGURE 40 MODEL UPDATER DPACKAGES I e Rene nene enne nene nn see en sn nein e ees enne nnn 76 FIGURE 41 TXT FILE SAMPLE EEN 76 FIGURE 42 CLOUDML ARTIST S HIGH LEVELSTEREOTVPES sese 82 FIGURE 43 CLOUDML ARTIST CORE DATA TYPES
25. has the following two main goals e Homogenization of the tool kit We aim at providing a homogeneous consistent tool kit for measuring the performance of cloud services An initial investigation about benchmarking tools 5 revealed that existing benchmarking tools are very heterogeneous concerning their distribution methods installation configuration and execution as well as their approach to collecting and formatting their results With the ARTIST s Benchmarking Suite we aim at homogenizing as much as possible the work flow for executing a benchmark from the download of the tool to the collection of results This will ease and enable the automation of execution of benchmarks Where possible existing third party benchmarking approaches and tools will be used to collect performance data There exist excellent tools that became de facto standards in the benchmarking field e g YCSB 6 for databases benchmarking We want to include these tools in the ARTIST s Benchmarking Suite because a they have been proved to work fine b they are supported by a large community of experts c there is a lot of documentation and tests already carried out and performance data already available and d users are already familiar with them e Automation The Benchmarking Suite is designed to automate as much as possible and for those tests that allows it the execution of testing tools Automation aspect is very important in this context because benchmarking to
26. measured by 3ALib Both Figure 48 and Figure 49 illustrate the total description of the overall availability SLA definition profile The UML profile describes SLA definitions that come from compute storage and platform level application frameworks layers and can be used to describe the specific SLA instances that are offered by the examined providers Google Amazon and Azure The rationale behind the abstraction of the SLA definitions is the identification of common concepts in the way the SLAs are defined for the respective providers Project Title ARTIST Contract No FP7317859 www artist project eu Page 106 of 129 D7 2 2 Cloud services modelling and performance Version v1 0 Final Date 15 09 2014 analysis framework Stereotype SLAAbstractedDefinition Compensation DiscountType QuantumOfTimelnMinutes Integer 1 Calculation CyclelnDays Integer 1 StatusAcknowledgement StateDiscovery 1 GuaranteedPercentage Real 1 ServiceName String 1 Stereotype Stereotype PaaSPro seri edis efinition IaaSProviderAbstractedDefinition StorageProviderAbstractedDefinition InstanceNumber Integer 1 ErrorRateLimitPerCent Integer 1 MultipleAZ Boolean 1 BackOffinterval BackOfflntervalDefinition 1 RestartFromTemplate Boolean 1 Preconditions PaaSOptions 1 Figure 48 General information of SLA definition regarding PaaS laaS and Storage providers The SLAAbstractedDefin
27. model generation process is depicted in Error No se encuentra el origen de la referencia WP7 will define the Cloud target metamodel CloudML Q ARTIST and will use it to instantiate the provider models in order to describe services and features Furthermore information from the Benchmarking Suite will be used to populate the performance fields of the provider models The detailed results will also be directed towards a central DB from which users will be able to query based on their interests by using a Web GUI Models will be included also in the ARTIST Repository in order to be considered in the related Work Packages of ARTIST such as Work Package 5 for the initial estimation e g by exploiting elements of cost Work Package 9 for the final selection of the offering based on all the needed criteria e g constraints performance attributes and Work Package 11 for the model level validation and testing of the migrated application WP5 Cost and feasibility assessment Use model Target info Metamodel Ue WP10 WP9 Target Fa Instantiate Repository selection Provider Stored in Model diuo Simulation preparation eN Benchmarki ng Suite Specialize d Queries Raw Data Web GUI DB Project Title ARTIST Contract No FP7 317859 www artist project eu Page 18 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework Figure 1 Overall clo
28. public private public or private Enumeration PricingPlans by_month year_discount pay_as_you_go_or_monthly other pay_as_you_go Enumeration by hour Certifications pci sas 70 pci and sas 70 other none Enumeration ServiceTypes Infrastructure Platform Software Version v0 6 Final Date 31 03 2015 ApplicationTypes DB Data Analytics Web Serving Streaming Software Testing WebSearch web2 0 Scientific Enumeration UsageTypes Free Paid Premier Stereotype MeasurementT test value Real 0 1 periodicityOfTest Real 0 1 average Real 0 1 iterations Integer 0 1 standardDeviation Real 0 1 coefficientOfVariation Real 0 1 higherIsBetter Boolean 0 1 De Enumeration HighLevelEvaluatio poor average extensive Stereotype MetricAndValue Stereotype PerformanceSLAM etric typeOfOperation String 1 guaranteedValue Real 1 higherlIsBetter Boolean 1 metricUnit String 1 referencelInterval Real 0 1 intervalUnit String 0 1 acceptedVariation Real 0 1 metric String 1 value String 1 Stereotype Scalabili Stereotype autoscale Boolean 1 defineMaxMinElasticVMs Boolean 1 Project Title ARTIST T De scalabilityCapabilities Scalability Capabilities Type 1 scalabilityMetrics ScalabilityMetricsType 1 Enumeration AvailabilityZones East_US North_Central_US Sou
29. repository is on GitHub at the following link https github com artist project ARTIST tree master source Tooling pre migration benchmarking9620controller e The structure of the delivered artefact is simple it is possible to identify four main folders and three files e benchmarks folder such folder contains the workloads configuration files and scripts useful for the execution e cloud providers such folder contains an example of provider s configuration values as the image to use or the platform will define the target environment object of benchmark e src such folder contains the core source code e test this folder contains a single file that allows the user to choose between the main function to execute e LICENSE such file contains information about the Apache 2 0 license e README such file contains the characteristics of this version of the released tool 5 1 Package information Concerning the benchmarking tools used in most cases they have been re packaged respecting licenses constraints of the single tools and made available through the ARTIST repository In order to address installation requirements on all platforms packages have been created in three formats rpm for RedHat based platforms deb for Debian based platforms and tar gz for all the others platforms e g Windows Although it required an initial effort to re package the tools this makes it much easier for users to install and use benchmarkin
30. scaling out services and so on WindowsAzureProcessingOffering ProccesingOffering stereotype defined at CloudMI Artist s core level is applied to this entity It contains the enumeration of services related with the processing capacity offered by the provider and the calculated sla for this kind of offering WindowsAzurePlatformOffering PlatformOfferingstereotype defined at CloudMl Artist s core level is applied to this entity It contains the enumeration of platform related service and the calculated sla for this kind of offering WindowsAzureStorageOffering StorageOffering stereotype defined at CloudMI Artist s core level is applied to this entity It contains the enumeration of services related with the storage capacity offered by the provider and the calculated sla for this kind of offering WindowsAzureProcessingService laaSProccesingService and laaSProviderAbstractedDefinition stereotypes areapplied to this entity It contains information related to the processing capacity offered such as maximum number of processors ram and processor architectures available It also contains availability information expressed by means of the application of the laaSProviderAbstractedDefinition sterotype contained in the Availability uml profile please see a complete description of the entity at Appendix B WindowsAzureServiceBus Entity created explicitly for Windows Azure provider Even though the service bus is offered also by other clou
31. support is highly needed for this task This may be achieved by pure static analysis of the system and the used services e g by analysing the models structures An alternative approach is to have dynamic analysis in terms of simulations to determine the best configurations and deployments Thus in order to achieve a meaningful and automated deployment space exploration process detailed information of Project Title ARTIST Contract No FP7 317859 www artist project eu Page 21 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework Input Tracing and Analysis Output Migrated __ Trace NC Execution ME Property l Extractor wee Analyzer e e 7 e f Migrated m U ie I og i s i gt I D Softwa 1 P Cloud Workload Model s 2 I See Goal Goal Goal Goal Ir Goal Achievement Evaluator BB 1 Evaluation Model Decomposition Condition e Report Goal Decomposition Goal Evaluation the non functional properties of the offered Cloud services is needed on the model level in order to use this information in static analysis algorithms or simulators 2 3 Figure 2 Overall methodology for evaluating non functional goals 1 4 4 Usage of cloud models in product testing and validation The task of testing and validation in Work Package 11 is to evaluate whether the specified functional and non functional goals of the migration have been achieved Fu
32. the configuration complexity of distributed applications in Internet data centers Communications Magazine IEEE 2006 Project Title ARTIST Contract No FP7 317859 www artist project eu Page 79 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework 20 DeltaCloud Drivers Online Available http deltacloud apache org drivers html 21 JClouds What is Jclouds Online Available http jclouds incubator apache org documentation gettingstarted what is jclouds 22 LibCloud Getting Started Online Available http libcloud apache org getting started html 23 Cobiacomm How to pick an ESB An Enterprise Service Bus Evaluation Framework Online Available http blog cobia net cobiacomm 2012 04 18 how to pick an esb an enterprise service bus evaluation framework 24 Chris Haddad electing an enterprise service bus Online Available http de slideshare net wso2 org enterprise use case selecting an enterprise service bus 25 Fiorano Online Available http www fiorano com docs 4 Comparision Fiorano BEAAquaLogic pdf 26 OpenLogic Online Available http de slideshare net OpenLogic comparison of open source esb solutions 27 Manrivo Online Available http www manrivo com data esbCompare061018 pdf 28 MPhasis Online Available http www mphasis com pdfs enterpriseservicebus pdf 29 Y Yahoo Overvi
33. the model parser is triggered and performs the requested changes The type of information given to the parser is name of provider name of benchmark type type of instance type upon the benchmark test has been performed set of properties measured for this instance and this benchmark type set of values corresponding to these properties The parser apprehends this information in modelling language terms the name of the UML file to be accessed the name of the benchmark stereotype applied on an InstanceType stereotype of the above provider s profile the name of this instance stereotype the set of properties of the benchmark stereotype to be accessed the set of values to be stored 8 2 1 Technical Specification The Model Updater is based on open source software and it has been developed using Eclipse environment In particular we are using the following java libraries for releasing the current components e UML2 e mysql connector java 5 1 34 bin jar GPL 2 0 License e guava 18 0 jar Apache 2 0 License Project Title ARTIST Contract No FP7 317859 www artist project eu Page 75 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework 9 Model Updater Delivery and Usage 9 1 Package information Model Updater is distributed as a jar file containing the source code and dependencies The file structure of the delivery package is shown in Figure 40 eu artist
34. www artist project eu Page 9 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework Table of Tables TABLE 1 VM INSTANCE CHARACTERISTICR eee en eereneesese sese sese reste sete sees este tetas e seen 53 TABLE 2 BENCHMARKING SUITE LICENSES cccceccscecccceccccecccecccsccescecsescecescecescucscssscecesescesesces 58 TABLE 3 AVAILABILITY EXPERIMENT RIATISTICR ee eere esee esee seres tes sese restare se eae 63 TABLE 4 SAMPLE STATISTICS FOR THE AMAZON SLA EXPERIMENT US bast 64 TABLE 5 INDICATIVE APPLICATION OF THE SLA STRICTNESS METRIC ON EXISTING PUBLIC CLOUD e 67 TABLE 6 3ALIB ACCEPTED PROVIDER NAMES rerernerernenernenennenennenennenennenennenennenennnnennnnensnnensnnenennenennenennen 70 Project Title ARTIST Contract No FP7 317859 www artist project eu Page 10 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework Terms and abbreviations AGPL Affero General Public License APT Advanced Packaging Tool Advanced software based seRvice provisioning and ARTIST migraTlon of legacy SofTware AWS Amazon Web Service BSD Berkeley Software Distribution CDDL Common Development and Distribution License CloudML Cloud Modelling Language CPU Central Processing Unit DataBase DRAM Dynamic Rando
35. 1 0 0167 normalized 0 5 Microsoft 1 1 normalized 1 normalized 99 95 Azure 0 1 0 0167 normalized Compute 0 5 An example of x not being O would be the Azure Storage SLA where unavailability is also determined by response time limits to a variety of service calls Project Title ARTIST Contract No FP7 317859 www artist project eu Page 67 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework 7 3ALib Delivery and Usage 7 1 Package information 3ALib is distributed as a jar file containing the source code and dependencies It is also availa ble as an Eclipse Java project for extension of the code base 7 2 Installation instructions In order to run the jar file only Java 7 needs to be installed in the target machine and set as environment variable In order to be installed as an Eclipse project the standard import in structions of Eclipse need to be followed 13 The Apache Jclouds project needs to be also included as a dependency based on the instructions that can be found in 14 For the database part the steps are the following e Install MongoDB following the documentation instructions 15 e Start MongoDB shell with the following instruction optional mongo e Create a new DB with the name 3alib optional use 3alib e Create a new collection equivalent to table in SQL with the name log samples op tional db createCollect
36. 1 memory 3 75 Le e af ele A3 D lt fy AmazonEC2Diagram 25 E Properties 7 J E M1Mediuminstance Applied stereotypes UML jaaSInstanceT ype Stereotype Si M1Mediuminstance laaSinstanceType instanceCategory General Purpose region East US North Central US South Central US West US East Asia IE 8 al a operatingSystem CentOS RedHatEnterpnseLinux amp Debian OracleEnterpriseLinux WindowsServer SUSELinuxEnterprise Ubuntu openSUSELinux Cancel Comment IB laaSInstanceType from CoreProfile laaS yp Profile Appearance Rulers And Grid Advanced Figure 30 Benchmark profile is applied to the Amazon EC2 profile e Finally it is possible to assign the respective average values from the benchmarking process to the properties defined in the MiMediuminstance stereotype The average values can be retrieved from the local database and placed to the respective place holder of the uml profile in an automated way by using the Model Updater prototype that is described in Section 8 and 9 extensively Project Title ARTIST Page 55 of 129 Contract No FP7 317859 www artist project eu D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework 5 Benchmarking Tools Delivery and Usage The Benchmarking tool is public and the Git source
37. 1 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework Regarding the Cloud Test three new options have been introduced according to benchmarking tools used by the controller e Java Applications for DaCapo Suite tool and all related workloads Select VM Instance t1 micro gt Select Cloud Test Java Applications Select tool DaCapo Suite Select workload avrora M Figure 12 Cloud test for DaCapo e Filesystem for Filebench tool and all related workloads select VM Instance tl micro M Select Cloud Test Filesvstem M Select tool Filebench KR Select workload yarmail M Figure 13 Cloud test for Filebench e YCSB for database to test and all related workloads select VM Instance tl micro J Select Cloud Test YCSB J Select Database MonaoDB ii select Workload Workload A j Figure 14 Cloud test for YCSB An example of output is shown in Figure 15 Itis related to DaCapo historical data filtering and consists of e atable describing the test number the Cloud Provider the target Platform the Date of execution and the response time milliseconds in this case e anindicator of average for the performance results it could be more than one in case of more performance data Project Title ARTIST Contract No FP7 317859 www artist project eu Page 42 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31
38. 317859 www artist project eu Page 34 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework A n aigesPesturese st Palett aSteregtypes FE lag Bag ies Mew Cloud Provider Features E e elaasFeaturess B Packs virtual private doud offering feke M Pac hynerysor Ven i pricinaPlan by month y ar_decount pay as you qo pe o m L emm a Cm eres ms d Classil IProviderDiagsam 25 n t oud Provider Features Segoe Ul x Font height 9 B L Description Shape Decoration Visible true false Position Tap Left tereotype display Text Text alignment Horizental Display place Compartment ppeled stereotyper 0 laasFeatures from CoreProfilelaa5 CS virtual private cloud eleng Boolean 1 1 false G hypervisor Hypervisars 1 1 Xen FY nmerinalEsn BrirannBPlans 1 he munnth wear dernan naw ss we nol Following these steps it is possible to model the main characteristics of existing IAAS providers by adding new stereotypes to the diagram and giving values to the needed parameters The same process can be followed to create profiles for PaaS providers As can be seen the metamodel can be extended in an easy way After adding new providers profiles it will be regenerated the eclipse plugin so these profiles will be able to be imported and applied in deployment models WP9 3 3 Licensing informati
39. AE MultiAZOption Amaz For Amazon multi AZ onRDS deployment Is automatic following the enablement of the according option by the user TotalCallsIncludedinU Factor of total calls ptime Azure SQL used in formula for defining uptime BackOfflntervalDefinition enumeration properties are shown in table This field includes the different logic and values defined by cloud providers regarding back off interval definition For Amazon this metric is not required thus we can set it at 0 while for Azure and GAE follows a dynamic calculation At the moment BackOffintervalDefinition includes enumeration literals Project Title ARTIST Contract No FP7317859 www artist project eu Page 109 of 129 D7 2 2 Cloud services modelling and performance Version v1 0 Final Date 15 09 2014 analysis framework named values however in the future we are going to extend this field in order to be linked with subtypes describing extensively the specific calculation of back off interval for each cloud provider Amazon Not necessary set it at 0 GAE Starting at 1 second and increasing exponentially up to 32 seconds Azure Azure after the first timeout server busy wait 3 seconds then 30 seconds and then 90 seconds before retrying after each subsequent timeout server busy StateDiscovery enumeration parameters are shown in the following table Each cloud provider provides in a different way information regarding the state of avail
40. BENCHMARKING GUI FILTERING OF HISTORICAL Data 40 FIGURE 10 RETRIEVAL OF INFORMATION AND DEPICTION e eee enn nennen nnne nene 4 FIGURE 11 INSTANCETYPES FOR CLOUD PROVIDER eeeee eee ee nennen nnne nnne nnne 4 FIGURE 12 CLOUD TEST FOR DAC APO E 42 FIGURE 13 CLOUD TEST FOR FILEBENCH eee ee ene nennen nhsnn nne se ne e e nsns e ense e anres 42 FIGURE 14 CLOUD ERSCH NGEN eene Se 42 FIGURE 15 GUI CONTROLLER INTEGRATION ee II Rene hene nnne nnne nne n e nsn sn seen nnns 43 FIGURE 16 BENCHMARK DATABASE SCHEMA I eee e menn nhee e nene nnne nn sen se sese ense enne nan 44 FIGURE 17 EXAMPLE OF BENCHMARK DATABASE VIEW 44 FIGURE 18 EXAMPLE OF REST INTERFACE FOR THE DATABASE eene nennen 45 FIGURE 19 YCSB ARCHITECTURE cccccccccccsccceseccceecececccceucccuueceeececeecceeeeceuseceuecsseceseneceeeeceuesenes 46 FIGURE 20 YCSB OUTPUT EXAMPLE ccccccseccssscccasscnecscscescscucsevacsenecacsenscscucsenacseneacsevacscussenacsenevacses 47 FIGURE ED NSR 5 25 9399909210800 M PIER a sea ERI d sad UDOPR TORDEEMCZUDDURT RIO SNAN CUR DONNE UPS aai 48 FIGURE 22 DWARFS STATISTICS vvs RsvU Pa EU RREE 48 FIGURE 23 INDIVIDUAL EXECUTION STATISTICS ruurernnrnnrnnnnrnnnnrrnnnnrnnnnrnnnnrnnnnernnnsnnnnnnnnnsnnnnsvennnsnnnnsnnnnssene 48 FIGURE 24 CLOUDSUITE DATA CACHING OUTPUT ESAMPLE eee e eene ener nennen 49 FIGURE 25 FILEBENCH OUTPUT EXAMPLE eene nemen e he ense hene rese rise ise eser sese e
41. BLE stopped 22 Array 0 Array 0 us east 1 ami 10 Keys d 5 Keys m us east 1 i 9a NODE 53986d7c838ec UNAVAILABLE stopped 22 Array 0 Array 0 us east l ami 10 Keys m 5 Keys sp us east 1 i 6a NODE 53986d7c838ec UNAVAILABLE stopped 22 Array 0 Array 0 us east l ami 10 Keys amp 5 Keys m us east 1 i 9a NODE 53986f91aa8fc UNAVAILABLE stopped 22 Array 0 Array 0 us east l ami 10 Keys gt 5 Keys amp us east 1 i ba NODE Figure 35 3ALib back end DB view 7 3 User Manual 7 3 1 Configuration 3ALib includes a configuration file 3alibconfig properties that is automatically stored in the top level folder of the project The file contains critical information on numerous issues such as provider keys IP for the log database etc The user must edit this file in order to include their own specific information The form of the file is as follows Mon May 12 20 34 05 EEST 2014 DBuser DBKey APIkey YOUR API KEY user YOUR USER KEY databaselP 127 0 0 1 ProviderName aws ServiceName ec2 Figure 36 3ALib Configuration file Values for APIkey and user are given following the authentication methods of Cloud providers DatabaselP is the IP of the server that contains the MongoDB instance for storing logging in formation and DBuser and DBkey the access credentials for this DB Provider and service names include the valid names as used by the library For the three supported providers this
42. D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework ARTIST FP7 317859 artist Advanced software based seRvice provisioning and migraTlon of legacy Software Deliverable D7 2 3 Cloud services modelling and performance analysis framework Nunzio Andrea Galante Gabriele Giammatteo Responsible Partner ENG Status Version Final v 0 6 31 03 2015 Distribution level CO PU PU Project Title ARTIST Contract No FP7 317859 www artist project eu Page 1 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework Project Number FP7 317859 Project Title ARTIST Title of Deliverable Cloud services modelling and performance analysis framework Due Date of Delivery to the EC 31 03 2015 Vor PACKAge eS pOnSIDIe OF py Meta modelling for target definition and the Deliverable cloud delivery Engineering Ingegneria Informatica S p A Contributor s ATOS Fraunhofer ICCS INRIA TECNALIA TUWIEN SPIKES Approved by All Partners Recommended mandatory readers WP5 WP7 WP9 WP10 WP11 This deliverable is threefold aiming initially at the validation of stereotypes and creating and incorporating the modelling framework for describing laaS PaaS providers and their attributes in the ARTIST platform and highlighting designing and implementing the benchmarking of selected PaaS laaS an
43. DATE 99thPercentileLatency ms Avg lat READ Operations e DOE READ Averagelatency ms 9sth READ MinLatency ms 99th R M i EAD MaxLatency ms v Bi Dwarfs v Bi rilebench Standard Deviation 6 READ 95thPercentileLatency ms w d e Mi Y Lei Columns v Lei Columns Min READ 99thPercentileLatency ms e e CloudProvider CloudProvider a INSERT Operations e User User AvgGetSize INSERT Averagelatency ms e Tool Tool RpsParameter INSERT MinLatency ms e Workload Workload Connections INSERT MaxLatency ms e Platform Platform Cpus INSERT 95thPercentileLatency ms e Number of runs Ops QoS_ms INSERT 99thPercentileLatency ms p on e SCAN Operations Structured Grid Scores OpsPerSecond e DaCapo SCAN s amant ms MapReduce MonteCarlo Score ReadWrite v DR Columns SCAN MinLatency ms Dense Matrix Scores MbPerSecond CloudProvid n oudProvider 9 SCAN MaxLatency ms Sparse Matrix Scores 9 CpuOperations i User SCAN 9SthPercentileLatency ms Graph Traversal Scores Latency SCAN 99thPercentileLatency ms Particle Scores gt We READ MODIFY WRITE Operations es gtt Eesen Workload READ MODIFY WRITE Averagelatency n e P 6 Platform READ MODIFY WRITE MinLatency ms MITES SCOTUS PerformanceTime ms READ MODIFY WRITE MaxLatency ms DryStone Scores Figure 16 Benchmark database schema Filter 1 A Export Eg Autosize FA 5 CloudProvider User Tool Workload Plat
44. J Profile UML Profile application Comments z 4 hs Hare Location Version Definition Profile Appearance Advanced Project Title ARTIST Contract No FP7 317859 www artist project eu Page 31 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework eh Apply Profiles TREE iB core profile uml Filter E Seek lc eu artist migration tes chot euartist migration tes clot gt main profiles f amazonec2 profile 8 core profile uml ib googleappengine r EN e e KE e e In next screen it has to be selected laaS profile and laaSDatatypes as the cloud provider to be modelled is of laaS type a P CoreProfile OE CoreDataTypes laaS EI lag E Daas platform resoi EI PaaSDataTypes e Once the laaS profile has been included it is time to model the provider by including stereotypes describing it characteristics e As can be seen in next screenshots Stereotypes can be dragged from the palette to the diagram and after this it is possible to apply higher level stereotypes to them at laaS level so it is not necessary to define again common properties In the example it is applied aaSProvider stereotype to the new created entity Palette kee o Packages EI Package Profile Stereotypes New Cloud Provider Comment l o Classifiers Stereotype E Import Metaclass Enumer
45. LiveUntilCrash Double 1 Time takes until crash pingingDelay Double 1 Delay because of pinging e g ping flood sizeofAttack Float 1 Attacks size Packet Per Second average Bits Per Second Project Title ARTIST Contract No FP7317859 www artist project eu Page 123 of 129 D7 2 2 Cloud services modelling and performance Version v1 0 Final Date 15 09 2014 analysis framework Appendix F Provider model example In the next two pictures a Windows Azure model is portrayed created by using CloudML Artist For doing this an uml model has been created that makes use of the previously described profiles Even it is an indivisible element the model has been split in two images for a better visualization The first one contains the main stereotypes describing the provider and the second one contains the stereotypes representing the instance types Windows Azure can offer Project Title ARTIST Contract No FP7317859 www artist project eu Page 124 of 129 D7 2 2 Cloud services modelling and performance analysis framework Version v1 0 Final Date 15 09 2014 laasFeatures Stereotype WindowsAzureFeatures IaasFeatures virtual private cloud offering true hypervisor Hyper_V pricingPlan by month year discount pay as you go year certification sas 70 scaleup true scaleout true monitoring extensive support average api extensive freeTier true typeAppOriented DB Web
46. P7 317859 www artist project eu Page 29 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework a New Papyrus Project Papyrus Project Create a New Papyrus Project Project name New Cloud Provider Use default location Location DA TECNALIA soft eclipse modeling kepter workspace New Browse E New Papyrus Project Select language of the diagram Diagram Language e UL SysML a UML e n Profile e Create an UML Profile Diagram Project Title ARTIST Contract No FP7 317859 www artist project eu Page 30 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework New Papyrus Project Initialization information Select name and kind of the diagram Diagram Name NewCloudP roviderDiagram Select a Diagram Kind Gay UML Profile Diagram You can load a template V A UML profile with basic primitive types ProfileWithBasicTypes profile Remember current selection Make sure to work from the Papyrus perspective within Eclipse environment In Model Explorer view we can see all the components that make up the just created diagram Apply the core profile profile to the Diagram by pressing button on Diagram s Properties Profile tab 7 Properties 23 d Model Validation 19 64 Repositones SB Query Execution deprecated E
47. Serving Web2_0 scope public availabilittyZones East US North Central US West US East Asia Southeast Asia Brazil South gpulnstances true loadBalancing true supportsOCCI true supportsMultitenancy true supportsFederation true discountType Stereotype WindowsAzureComputeSLADiscount DiscountType MinPercentage 0 0 MaxPercentage 98 999 Discount 25 0 iaasProvider Stereotype WindosAzure IaasProvider iaasFeatures WindowsAzureFeatures instanceTypes AO A2 A1 A4 AS A6 A3 A7 A8 A9 description 2 Microsoft Azure iaas provider homepage http azure microsoft com processingOffering WindowsAzureProcessingOffering storageOffering WindowsAzureStorageOffering plafformOffering WindowsAzurePlatformOffering processing Offering Stereotype WindowsAczureProcessing ProcessingOffering service WindowsAzureProcessingService itaaSProcessingService iaaSProviderAbstractedDefiniti Stereotype WindowsAzureProcessingService IaaSProcessingService numProcessors 16 memory 112 platform CentOs RedHatEnterpriseLinux Debian OracleEnterpriseLinux WindowsServer plafformOffering Stereotype WindowsAzurePlatform Offering PlafformOffering service WindowsAzureServiceBus laaSServiceBus Stereotype WindowsAzureServiceBus IaaSServiceBus namespace namespace 1 region East US North Central US storage Offering Stereotype Win
48. WebSphere Application Server MS NET Application Container Jetty servlet container Go 1 www artist project eu InstanceTypes Backend Frontend Enumeration SalProducts MySQL PostgreSQL Mssqiserver Orade D7 2 2 Cloud services modelling and performance Version v1 0 Final Date 15 09 2014 analysis framework Next a detailed description of the most relevant sterotypes included in the PaaS profile PaaSDataStorageOffering Stereotype StorageOffering sla Real 0 1 ZN Stereotype PaaSDataStorageOffering datastoreAccessType DatastoreAccessType sqiStorageSeryite nonSalStorageService Stereotype Stereotype SalStorageService NonSalStorageService dataStructureType DataStructureTypes 1 type NoSQLTypes 1 This stereotype allows modelling the data storage offering related to a PaaS provider It extends from StorageOffering stereotype that is one of the common stereotypes that can be used in the context of PaaS and laaS providers models it is included at a higher level The data storage offering consists of a set of services that can be offered by the provider These services are classified as sql storage services and non sql storage services depending on the nature of the storage system provided Next are described the properties contained in the stereotype Name Type Card Description datastoreAccessType DatastoreAccessType Li T
49. ability vane NN Connection Simple connection ability to the VM Ping Ping utility to verify availability BoundedResponseTi Determination of a response time limit which characterizes if the service Is available or not Project Title ARTIST Contract No FP7317859 www artist project eu Page 110 of 129 D7 2 2 Cloud services modelling and performance Version v1 0 Final Date 15 09 2014 analysis framework DiscountType parameters are shown in the following table This includes the availability levels between which a discount applies along with the percentage of discount Name We 0 ears MinPercentage 1 Minimum percentage of actual observed availability value for which the discount applies MaxPercentage 1 Maximum percentage of actual observed availability value for which the discount applies Discount 1 The amount of discount offered by cloud providers in case of not meeting SLA agreements regarding availability Project Title ARTIST Contract No FP7317859 www artist project eu Page 111 of 129 D7 2 2 Cloud services modelling and performance Version v1 0 Final Date 15 09 2014 analysis framework Appendix C Performance Profile As mentioned in D7 1 in order to identify and check the performance rating across different application areas we have defined application types that correspond to various common appli cations and can be met in real life environment These application types
50. ackage 5 forward engineering process Work Package 9 and the testing and validation process Work Package 11 will use the data Furthermore we have separated the average information contained in the provider profiles from the overall raw data in order to keep the former lightweight and usable The raw data are exposed through the benchmarking GUI to an interested party that is more focused in performance and can make dynamic and configurable queries towards the raw data DB Project Title ARTIST Contract No FP7 317859 www artist project eu Page 78 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework References 1 2 3 4 5 6 7 8 9 G Kousiouris and A Evangelinou Artist Project D7 2 PaaS laaS Metamodelling Requirements and SOTA F S ren F Fittkau and W Hasselbring Search based genetic optimization for deployment and reconfiguration of software in the cloud ICSE 2013 pp 512 521 D Cosmo Lienhardt Treinen Zacchiroli Zwolakowski Aeiche and Agahi Automated Synthesis and Deployment of Cloud Applications ASE2014 H Page CloudML Online Available http cloudml org Electrical Engineering and Computer SciencesUniversity of California at Berkeley Above the Clouds A Berkeley View of CloudComputing 2009 Y GitHub YCSB Getting Started Online Available https github com brianfrankcoop
51. an open source C frameworks that uses loadable workload personalities to allow easy emulation of complex applications We have tested the last version and it s is resulted quick to set up and easy to use Filebench includes many features to facilitate file system benchmarking e Multiple workload types support via loadable personalities Ships with a library of more than 40 pre defined personalities including the ones that describe mail web file and database servers behaviour Workload personalities define the workload to apply to the system they include tunables for scaling workloads to specific systems e Easy to add new personalities using reach Workload Model Language WML 11 e Multi process and multi thread workload support Project Title ARTIST Contract No FP7 317859 www artist project eu Page 49 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework e Configurable directory hierarchies with depth width and file sizes set to given Statistical distributions e Support of asynchronous I O and process synchronization primitives e Integrated statistics for throughput latency and CPU cycle counts per system call Tested on Linux FreeBSD and Solaris platforms The ARTIST Repository contains the version 1 4 9 of Filebench tool and the related test results carried out on Linux In the following an example Figure 25 of output concerning Fileserver
52. ance Version v1 0 Final Date 15 09 2014 analysis framework benchmark application type All of the above benchmark results included in the performance model are represented in Figure 51 Stereotype BenchmarkResult workload WorkloadType 1 Stereotype Stereotype Stereotype DaCapoResult 7 x P CloudSuiteResult runtime Real 0 score e a t Periormencetime Real 10 1 throughput Real 0 1 Size Real 0 1 throughputOpsSec Real 0 1 tage See Tea 10 2 operations Real 0 1 rw Real 0 1 averageLatency Real bandwithMbEc Real 0 1 minLatency Real 0 1 cpuOp Real 1 maxLatency Real 0 1 latency Real 0 1 Figure 51 Benchmark results included in performance profile In order to simplify the benchmark profile we have defined one universal enumeration Figure 52 that includes the default workloads from the aforementioned benchmark categories These workloads are static in order to be able to compare the performance of different services on the same examined workload Enumeration YCSB Update Heavy YCSB Read Heavy YCSB Read Only YCSB Read Latest YCSB Short Range Filebench Webserver Filebench Fileserver Filebench Varmail Filebench Videoserver Filebench Webproxy Filebench OLTP DaCapo_Avrora DaCapo_Batik DaCapo_Jython DaCapo_Lulndex DaCapo Xalan CloudSuite Datacaching Twitter Dwa
53. ans by Possible values Type of offered a cloud provider be PricingPlans can found on enumeration data type see on data types section In case it is necessary more detailed information regarding prices and discounts can be set by applying the cost uml profile see on Cost when profile section creating a specific provider s model Types of certifications provider be Certifications owned by a Possible values can found on enumeration type Indicates if the provider offers the capacity to scale up increasing resources on the same server Indicates if the provider offers the capacity to scale out increasing resources the by balancing load among servers Monitoring capabilities offered by the provider Values can be selected from HighLevelEvaluation enumeration type This indication property gives an the supporting level offered by of the provider Values can be selected Contract No FP7 317859 www artist project eu D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework o HighLevelEvaluation freeTier Boolean from HighLevelEvaluation enumeration type Indicates in general terms if the api offered by the provider to access its infrastructure permits to perform a lot of operations or offers quite limited access Indicates if the provider offers a free tier or not typeAppOriented App
54. ation Runtime services through which are offered different runtimes programming languages and application frameworks and Application container services offering the possibility of using different preinstalled application servers When modelling those services specific values for the properties i e programming languajes can be selected from the enumeration entities offered in the scope of the PaaS profile on the right side of the picture InstancesService stereotype Stereotype InstancesService PaasProvider instancesService type InstanceTypes 1 name String 1 memoryOffered String 1 cpuOffered String 1 Project Title ARTIST Contract No FP7317859 www artist project eu Page 100 of 129 D7 2 2 Cloud services modelling and performance Version v1 0 Final Date 15 09 2014 analysis framework Apart from providing platform related services PaaS providers also offer infrastructure resources so the applications can be deployed on them With the aim of modelling this feature the InstancesServices stereotype allows to model the different instance types a PaaS provider can offer By now and after analisyng the offerings of most relevant PaaS providers the instances can be classified as being of Backend or Frontend types Each instance offers a specific amount of memory and cpu available so the developer deployer can choose which one is the more appropriate according
55. ation di RelationShips Extension Generalization Link Project Title ARTIST Contract No FP7 317859 www artist project eu Page 32 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework eeu is I FM fennnrt Metarlass V Go EE g tion I Stereotypes 5 RelationShips New_Cloud_Provider A Extension Generalizatinn V c Features Property j I ii neratinn 7 viderDiagram 23 J Model Validation 1 amp 3 Git Repositories Qu ery Execution deprecated Provider pplied stereotypes D D name New Cloud Provider E laasProvider from CoreProfile IaaS E iaasFeatures laasFeatures 1 1 null E name String 1 1 New Cloud Provider E description String 1 1 null E homepage String 1 1 null IE processingOffering ProcessingOffering 0 1 null Ej storageOttering StorageUffering 0 1 null E networkOffering NetworkOffering 0 1 null e After doing this there can be assigned values to the properties defined in the selected stereotype aaSProvider in this case There are some properties whose type corresponds to one of the stereotypes that have to be defined in the profile for example iaasFeatures property in the case of New Cloud Provider el jaasFeatures Stereotypes Mew Cloud Provider Featur n roviderDiagram Jf Model Validation 5 Git Repositories Query Execution
56. atistics per test system time wall clock time CPU etc system information memory usage etc in the same reporting structure as Figure 21 Project Title ARTIST Contract No FP7 317859 www artist project eu Page 47 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework GNU nano 7 27 6 File dwarf scores csyv Figure 21 Dwarfs scores Also for each test performed there are detailed statistics with regard to its configuration and specific timings such as minimum maximum and average times standard deviation and coefficient of variation Figure 22 and the statistics for each individual run Figure 23 GNU nano 2 2 6 File MapReduce MonteCarlo stats csv Figure 22 Dwarfs statistics GNU nano 2 2 6 File quicksortOutput csv ABEE VIVI Vy 2UVUUYUUU yp Jr Lu Qe JIA GA J a he 2VVUUUUU y 4332 aw VU wt A he UIT A12 aw tu dd eg AVIV A22 Lu ul ke 2VUUUUL p AU LLU Figure 23 Individual execution statistics These results should be parsed and added to the appropriate structures of ARTIST e g ARTIST repository in order to aid in the selection process for the most appropriate offering performed by WP9 4 2 4 3 CloudSuite CloudSuite 9 is a benchmark suite for emerging scale out applications the suite consists of eight applications selected based on their data center popularity e Data caching e Media streaming e Data serving e G
57. automatize the creation of binary packages and software repository we decided to rely where possible on an automation tool named ETICS 7 which provides functionalities for creating distribution packages in various formats such as tar gz rom deb ETICS also provides a repository that satisfies the requirements listed above The ETICS repository even if not integrated in the ARTIST Repository will be used in this third release as benchmarking tools repository In next releases of the Benchmarking Suite tools will be either made available in the ARTIST Repository or the ETICS Repository will be integrated in the ARTIST Repository A possible integration scenario for the ETICS repository is depicted in Figure 8 The software packages are created and maintained with ETICS In periodic intervals the ETICS repository contents are scanned and corresponding artefacts are created in the ARTIST repository that contains the search relevant meta data and a link to the software package in the ETICS repository This approach makes the software packages available to packet management systems as well as the ARTIST tools Project Title ARTIST Contract No FP7 317859 www artist project eu Page 38 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework Benchmarking Repository ARTIST Repository managed created by ETICS TUE references eerie Package Package Metadata YUM Clien
58. bility of services regardless of the supported provider on which they are deployed The library may measure availability based on the respective providers SLA definitions and thus conclude for potential SLA violations including also the evidence of this violation It can also provide recommendations as to whether a specific service instance is viable for an SLA based on the provider s preconditions The fourth prototype consists of a tool to update the provider models profiles that are stored in the repository with new results coming from the benchmarking experiments While the overall data of the experiments are kept in an internal Raw Data DB and not stored in the pro vider profiles in order to keep the latter lightweight the average performance information contained in them needs to be periodically updated based on the new range of measurements conducted each time Furthermore In the third year of the project we focused also in the integration aspects and better coordination of the individual outcomes of each tool in order to offer a more unified experience to the tool users This integration covered e Raw performance data exposed through a GUI for configurable queries e Provisioning of the results to WP11 in the format needed by the GMF more info on this can be found in D11 3 2 e Linking of the profiles with the update in the experiments results e A detailed experiment on the usage of 3ALib on a 1 5 month usage of AWS Project Tit
59. bilityMetrics ScalabilityMetricsType 1 Stereotype ScalabilityCapabilities Type autoscale Boolean 1 defineMaxMinElasticVMs Boolean 1 Stereotype ScalabilityMetricsType bootTime MeasurementType 1 scaleUpTime MeasurementType 1 scaleDownTime MeasurementType 1 Project Title ARTIST Contract No FP7 317859 www artist project eu Page 82 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework Provider Stereotype Provider name String 1 description String 1 homepage String 1 processingOffering 0 1 networkOffering storageOffering 0 1 0 1 0 1 Stereotype Stereotype Stereotype Stereotype Stereotype ProcessingOffering StorageOffering NetworkOffering PlafformOffering SaaSOffering lafformOfferi pla ormO ering saaSOffe ing 0 1 This stereotype is used to specify general information of the provider to be modelled such as the name and homepage It is extended from stereotypes representing PaaS laaS and SaaS providers contained in the correspondent uml profiles see next sections so when modelling a provider it is possible to reuse these properties specified at this higher level and to include some others related to the nature of the provider Next are described the properties contained in the stereotype Be ce ee peemer mm name String 1 Unique identif
60. c services that are generalizations of common Cloud services in a first step and subsequently refine these selected generic services by finding the most appropriate offered services of Cloud providers Thus a catalogue of services offered by Cloud providers is needed to generalize the common services as well as to document the specific realizations and their associated quality Automated service selection In order to provide a higher level of automation in the forward engineering process instead of manually selecting abstract services and refining them to concrete services requirements may be specified that should be fulfilled by the systems or by components These requirements are the main driver for guiding architectural redesign decisions and are the foundation for a match making process for computing the best fitting set of services to fulfil these given requirements By this the user does not have to define how the system should be realized instead she can focus on what should be realized Thus the functional and non functional properties of the services have to be defined in a format that is automatically processable Deployment optimization Once a set of services is selected additional configurations and deployment options may be automatically examined in order to improve performance measures or costs Current Cloud platforms offer a huge set of different options that leads to an overwhelming search space for the user Thus automation
61. ce to address these aspects e g different storage services for archiving fast retrieval low cost etc A number of supporting services may exist that can indicate the flexibility for managing cloud resources and should be included in order for a developer to make an informed decision of the suitable cloud provider service based on her interests and intentions for manageability For example billing alerts advanced monitoring ability to project into the future automated scalability rules are characteristics that can be helpful to an application developer who intends to migrate an application to the cloud but may or may not be needed for a specific use case By having this information the application developer may select in an optimal manner the target platform based for example on different aspects such as cost or richness of supporting services The conceptual structure and hierarchical division of the CloudML ARTIST is depicted in 1 The main point to be aware is the Multitypes part This contains attributes that can be included in more than one aaS levels In order to reduce schema size and provider descriptions these characteristics may be divided in two categories 1 attributes that are generic with respect to the provider and do not need to be repeated in every offering of the latter e g billing granularity etc 2 attributes that may be different in form or content for each service offering and must be included in each case
62. d designing and implementing an abstract solution for the availability measurements of cloud service Keyword List Meta modelling CloudML ARTIST Benchmarking Availability Model updater Licensing information The reported code will be licensed under open source with following licenses Benchmarking under Apache v2 Project Title ARTIST Contract No FP7 317859 www artist project eu Page 2 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework 3ALib under Apache v2 ClooudML ARTIST under EPL Model Updater under EPL The document itself is delivered as a description for the European Commission about the released software so it is not public Project Title ARTIST Contract No FP7 317859 www artist project eu Page 3 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework Document Description Document Revision History Modifications Introduced Version Date Modification Reason Modified by 10 03 2015 Contributions integrated Project Title ARTIST Contract No FP7 317859 www artist project eu Page 4 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework Table of Contents TN 5 TRUE 8 TETT NN 10 Terms and abbreviations enne T s san nnn tense sss a serrer nennen 11 PENE AY TT Tm 13 fodere Ure
63. d providers for now it is not so extended to make it necessary to define it at a higher level in CloudML Q2 Artist This is why it has been included as a standalone stereotype in the model The stereotype contains information related to the namespaces where the service can be applied and also the regions where it is applicable WindowsAzureStorageService laaSStorageService is applied to this entity It contains information related to the storage capacity offered such as type of storage offered by the provider regions there may exist different data privacy constraints depending on the country where it is stored maximum capacity in gigabytes and replication types WindowsAzureComputeSLADiscount1 DiscountType stereotype from Availability Project Title ARTIST Contract No FP7317859 www artist project eu Page 126 of 129 D7 2 2 Cloud services modelling and performance Version v1 0 Final Date 15 09 2014 analysis framework profile see a detailed description on Appendix B is applied to this entity Main purpose of this stereotype is to represent the amount of discount offered by cloud providers in case of not meeting SLA agreements regarding availability Project Title ARTIST Contract No FP7317859 www artist project eu Page 127 of 129 analysis framework iaaSInstanceType Stereotype Ad IaaSInstanceType instanceCategory General region operatingSystem CentOS RedHatEnterpriseLinux Debi Oracl
64. dbuser String 4 dbpass String 4 apikey String 4 libUserID int Month String 4 quantumOfTimeinSeconds int setDbpass String void run void storeStatistics void getProviderName String setProviderName String void S Availabilty Audtor getServiceName String setServiceName String void 6 getUser String amp setUs er String void getApikey String setApikey String void A 9 getLibUs erfX int setLibUserD nt void amp getMonth String setMonth String void amp getQuantumOfTimeinSeconds int setQuantumOfTimeinSeconds int void getDohosi String amp setDbhost String void getDbuser String setDbuser String void getDopass String E X A lt lt Java Class lt lt Java Class lt lt Java Class lt lt Java Class 3MicrosoftAzureCompute SLAAuditor S AWSEC2SLAAuditor 9 GoogleCompute SLAAuditor 9 ANSEC2SLAAuditorRegion eu artist cloud auditors eu artist cloud auditors eu_artist cl i eu artist cloud auditors F AWSEC2SLAAudtorRegion amp checks pplicabiity Conditions String String String String String String void checkReachability NodeMetadata boolean addReachabiityInfo boolean NodeMetadata String AWSEC2SLAAudtor Micros oftAzureComputeSLAAuditor E GoogleComputeSLAAuditor amp checkA pplicability Conditions String String String String String String void j amp checkReachability N
65. decided to release the ARTIST Benchmarking Suite a tool kit for managing the entire benchmarking process in ARTIST Initial tests have been executed using these tools and feedback from the users has been used in order to update aspects of automatic execution and test setup In this version the ARTIST Benchmarking Suite used to execute the tests has been integrated with the Benchmarking GUI for viewing the results in a dynamic selection manner Availability aspects are covered by 3Alib library in particular it is an abstracted way to measure the availability of services regardless of the supported provider on which they are deployed Such library will be able to calculate the actual levels of availability as defined in the respective providers SLAs based on the latter s definition of the term This tool is expected to be of great help for individual users in order to validate their SLAs In this version we have extended the tool to include Google Compute Engine and we have contributed in the investigation of the incorporation of GAE services and mainly the Google Datastore Furthermore we applied the 3aLib on a 1 5 month experiment on Amazon AWS monitoring in order to investigate its robustness and the production of results Artefacts produced using the three prototypes are published in the ARTIST Repository and will be used as inputs by the others processes of a migration procedure In particular the business and technical feasibility Work P
66. deprecated id Provider Features Applied stereotypes D ERT E laasFeatures from CoreProfile TaaS gt E virtual private cloud offering Boolean 1 1 false El hypervisor Hypervisors 1 1 Aen z as you go d E pricingP lan PricingPlans 0 by month year discount pay as you go E certification Certifications 0 gt E scaleup Boolean 1 1 false E scaleout Boolean 1 1 false gt DI monitoring HighLevelEvaluation 1 1 poor mr k m by month year discount Project Title ARTIST Contract No FP7 317859 www artist project eu Page 33 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework Applicable Stereotypes Applied Stereotypes e a Stereotype laasProvider Stereotypes S New Cloud Provider IaasFeatures o Es laaSInstanceType 3 E 5 laaSProcessingService ee I In those cases before applying a value to the property first it will be necessary to include a stereotype implementing the needed stereotype here iaasFeatures stereotype After the stereotype s properties have been given values it is necessary to specify which ones of those properties have to be shown in the class diagram For this Appearance tab has to be clicked and after that next steps are select the properties to show and press first button on the right Project Title ARTIST Contract No FP7
67. dowsAzure Storage Offering StorageOffering service WindowsAzureStorageService laaSStorageService Stereotype WindowsAzureStorageService IaaSStorageService type Block blobs Page blobs Tables Queues region East US North Central US South Central US West US East Asia Southeast Asia Brazil South Project Title ARTIST Figure 54 Windows Azure model main stereotypes Contract No FP7317859 Page 125 of 129 www artist project eu D7 2 2 Cloud services modelling and performance Version v1 0 Final Date 15 09 2014 analysis framework As stated out before in the previous image can be seen the main stereotypes contained in Windows Azure model Next a brief description of each one WindowsAzure this entity implements laaSProvider stereotype It contains properties describing the provider in general terms such as name description and home page and other properties referencing entities in the model specifying Windows Azure s features WindowsAzureFeatures and the different offerings given by the provider WindowsAzureProcessingOffering WindowsAzurePlatformOffering WindowsAzureStorageOffering WindowsAzureFeatures laasFeatures whose description can be found on Appendix A is applied to this entity It contains specific values related to Windows Azure for properties containing information such as generic pricing plans offered availability zones if the provider offers scaling up and
68. duces the main extensions introduced in the CloudML QARTIST meta model compared to the original CloudML meta model Extensions are described grouped by functionalities in separate sub sections that present the context of the cloud characteristic analysed by the extensions and a rationale of the stereotypes design Apart from the extensions that can be applied in a selective way depending on the specific modelling needs there is a small set of stereotypes at a higher level containing generic entities and properties that can be applied to any cloud provider independently of the provider s nature laaS PaaS or SaaS Next a picture of this set of stereotypes and a description of each one Project Title ARTIST Contract No FP7 317859 www artist project eu Page 81 of 129 D7 2 3 Cloud services modelling and performance analysis framework pricingPlan PricingPlans certification Certifications scaleup Boolean 1 scaleout Boolean 1 monitoring HighLevelE valuation 1 support HighLevelE valuation 1 api HighLevelE valuation 1 freeTier Boolean 1 typeAppOriented ApplicationTypes scope PublicPrivate 1 availabiltyZones AvailabiityZones gpulnstances Boolean 1 loadBalancing Boolean 1 supportsOCCI Boolean 1 supportsMultitenancy Boolean 1 supportsFederation Boolean 1 usageType UsageTypes name String 1 description String
69. e instanceCategory Memory region operatingSystem CentOS RedHatEnterpriseLinux Debi OracleEnterpriseLinux Windd SUSELinuxEnterprise Ubunt FreeBSD openSUSELinux SUSEEnterpriseLinux virtualCores 8 memory 56 0 es 15 09 2014 iaaSInstanceType Stereotype A3 IaaSInstanceType instanceCategory General region operatingSystem Centos RedHatEnterpriseLinux Debi OracleEnterpriseLinux WindowsServer SUSELinuxEnterprise Ubunt FreeBSD openSUSELinux SUSEEnterpriseLinux virtualCores 4 iaaSInstanceType Stereotype A8 IaaSInstanceType instanceCategory Compute region operatingSystem CentOS RedHatEnterpriseLinux Debi OracleEnterpriseLinux Winda SUSELinuxEnterprise Ubunt openSUSELinux SUSEEnterp virtualCores 8 memory 56 0 localDisk 127 Figure 55 Windows Azure instance types Contract No FP7317859 Page 128 of 129 iaaSInstanceType Stereotype A4 IaaSInstanceType instanceCategory General region operatingSystem CentOS RedHatEnterpriseLinux Debi OracleEnterpriseLinux WindowsServer SUSELinuxEnterprise Ubunt FreeBSD openSUSELinux SUSEEnterpriseLinux virtualCores 8 iaaSInstanceType Stereotype A9 IaaSInstanceType instanceCategory Compute region operatingSystem CentOS RedHatEnterpriseLinux Debi OracleEnterpriseLinux Windd SUSELinuxEnterprise Ubunt openSUSELinux SUSEEnterp vir
70. e ARTIST Repository ready to be executed on target cloud environments e a repository for results that is part of the ARTIST Repository where results for all executions are stored for later consultation and querying e aGUlthat is used for configuring and submitting benchmark tests automatically when possible on remote cloud infrastructure and access results e acontroller component that will orchestrate the execution of benchmarks and manage the collection of results from tools output and publication on the ARTIST Repository The third prototype released by Work Package 7 is the 3Alib software the main objective of such library is to measure the availability of services using an abstracted approach regardless of the supported provider on which they are deployed Such library will be able to calculate the actual levels of availability as defined in the respective providers SLA The goals of the library are e consult users on why their services do not adhere to the SLA if applicable e keep detailed logs of the availability of services Project Title ARTIST Contract No FP7 317859 www artist project eu Page 15 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework e calculate availability levels based on the providers definition and be used for compen sation claims together with the aforementioned logs The fourth prototype is a Model updater tool which is used in
71. e profile UML Profile containing generic stereotypes and data types that can be applied to characterize entities belonging to different cloud providers As can be seen in figure 4 the Core profile is divided into 3 sub profiles for a better understandability and usage e laaS contains specific laaS stereotypes and data types As it is a sub profile contained in the Core Profile the stereotypes contained in it can extend directly those stereotypes common stereotypes defined at a higher level and can also make use of the common data types at that level This is applicable also to PaaS and SaaS subprofiles e PaaS contains specific PaaS stereotypes and data types e SaaS contains specific SaaS stereotypes and data types Amazon EC2 profile Such profile describes Amazon EC2 provider and allows to create models to specify values for concrete deployments on this provider As Amazon EC2 is an laaS provider this profile imports laaS profile and makes use of generic stereotypes defined at that level in the same way any other laaS provider could do The use of this inheritance mechanism is very convenient in order to not repeat the creation of stereotypes that have been defined at a higher level WindowsAzure profile UML profile with same purpose than Amazon EC2 but oriented to specify Azure s characteristics Project Title ARTIST Contract No FP7 317859 www artist project eu Page 27 of 129 D7 2 3 Cloud services modelling and p
72. eEnterpriseLinux WindowsServer SUSELinuxEnterprise Ubunt FreeBSD openSUSELinux SUSEEnterpriseLinux ores 0 iaaSInstanceType Stereotype AS IaaSInstanceType instanceCategory Memory region operatingSystem CentOS RedHatEnterpriseLinux Debi OracleEnterpriseLinux Winda SUSELinuxEnterprise Ubunt FreeBSD openSUSELinux SUSEEnterpriseLinux virtualCores 2 memory 7 14 0 c NE Project Title ARTIST D7 2 2 Cloud services modelling and performance iaaSInstanceType Stereotype Al IaaSInstanceType instanceCategory General region operatingSystem CentOS RedHatEnterpriseLinux Debi OracleEnterpriseLinux WindowsServer SUSELinuxEnterprise Ubunt FreeBSD openSUSELinux SUSEEnterpriseLinux virtualCores 1 iaaSInstanceType Stereotype A6 IaaSInstanceType instanceCategory Memory region operatingSystem CentOS RedHatEnterpriseLinux Debi OracleEnterpriseLinux Winda SUSELinuxEnterprise Ubunt FreeBSD openSUSELinux SUSEEnterpriseLinux virtualCores 4 memory 28 0 Version v1 0 Final Date iaaSInstanceType Stereotype A2 IaaSInstanceType instanceCategory General region operatingSystem CentOS RedHatEnterpriseLinux Debi OracleEnterpriseLinux WindowsServer SUSELinuxEnterprise Ubunt FreeBSD openSUSELinux SUSEEnterpriseLinux virtualCores 2 iaaSInstanceType Stereotype AT IaaSInstanceTyp
73. eady mentioned in section 2 profile for Amazon EC2 laaS cloud provider has already been created and included in the CloudML ARTIST model e Apply the benchmark profile to the Diagram by pressing button on Diagram s Properties Profile tab The aforementioned uml profile is stored in the supporting profiles folder Figure 29 7 Apply Profiles EI i i8 benchmark profile uml Filter 4 1 eu artist migration tes cloudmlartist 2 main profiles 4 supporting profiles amp availability profile uml benchmark profile um pricing profile uml d G iB security profile uml teststore U ti i UML Profiles profile uml i ken Leg Figure 29 Benchmark profile is applied to the Diagram e Next step includes the Appliance of the benchmark profile to the Amazon EC2 profile Project Title ARTIST Contract No FP7 317859 www artist project eu Page 54 of 129 D7 2 3 Cloud services modelling and performance analysis framework Version v0 6 Final Date 31 03 2015 by pressing button on Properties Profile tab To be more precise DaCapoResult FilebenchResult and YCSBResult Stereotypes from benchmark profile should be applied to one of the laasInstanceType Stereotypes of Amazon EC2 profile For instance we can follow this process for MiMediuminstance Stereotype 7 amazonec profile di 23 R North Europe SUSEEnterpriseLinux vitualCores
74. ed descriptions of the platform or infrastructure used for the deployment This is especially true for the aspects concerning cloud resources the physical infrastructure or scalability metrics Furthermore the 3ALib library may also be used directly for measuring a specific deployment s availability service levels 1 4 5 Integration with the other Artist repository In order to finalize the integration between the raw data DB and the provider models in the repository an extra step is necessary through the Model Updater tool The purpose of this is to Project Title ARTIST Contract No FP7 317859 www artist project eu Page 22 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework be run periodically and update the performance values that exist in the provider models Given that the models should be as lightweight as possible it was decided to provide in them information regarding the average values of a provider s service type e g VM size type on the higher possible level of benchmark abstraction Thus the categories in this case would be as generic as e Java benchmarks average score e Filesystem benchmarks average score e Databases benchmarks average score lf a user would need more fine grained information e g Java benchmark on a specific workload for a specific period in time then they should forward their request through the Raw Data Db in Figure 1 Pr
75. emulator benchmark root user go filebench Filebench Version 1 4 9 12324 0 000 Allocated 170MB of shared memory filebench gt load fileserver 12462 2 869 FileServer Version 2 2 personality successfully loaded 12462 2 869 Usage set dir lt dir gt 12462 2 869 set Smeanfilesize lt size gt defaults to 131072 12462 2 869 set Snfiles lt value gt defaults to 10000 12462 2 869 set nthreads lt value gt defaults to 50 12462 2 869 set Smeanappendsize lt value gt defaults to 16384 12462 2 8659 set Siosize lt size gt defaults to 1048576 12462 2 869 set Smeandirwidth lt size gt defaults to 20 12462 2 869 sets mean dir width and dir depth is calculated as log width nfiles 12462 2 869 run runtime e g run 60 filebench set dir mnt filebench gt run 60 12462 4 909 Creating pre allocating files and filesets 12462 4 918 Fileset bigfileset 10000 files avg dir width 230 avg dir depth 3 1 12740 757MB 12462 5 280 Removed any existing fileset bigfileset in 1 seconds 12462 5 280 making tree for filset tmp bigfileset 12462 5 290 Creating fileset bigfileset 12462 6 080 Preallocated 7979 of 10000 of fileset bigfileset in 1 seconds 12462 6 080 waiting for fileset pre allocation to finish 12466 6 080 Starting 1 filereader instances 12467 6 081 Starting 50 filereaderthread threads 12462 7 137 Running 12462 67 142 Run took 60 seconds 12462 67 145 Per Operation Breakdown sta
76. entified viola tion interval It is not determined in the SLAs of the providers whether the discount in case of violation ap plies to the entire account amount or to the part valid template IDs that is applicable to an SLA However given that this is a dynamic list that may vary over time and that providers report the overall amount for a type of resource calculating the potential discount in case of partial discount can be a very challenging if not impossible due to lack of necessary information task One key consideration is the incorporation of checks regarding connectivity at the client side This is extremely important since in case of poor connectivity at the client side the samples may appear as unreachable Thus once a sample is considered as unreachable an extra check must be performed on the client e g pinging of a well known address like Google DNS at 8 8 8 8 in order to ensure that it is the service side that is unresponsive Another key consideration is the analysis per regions since at that level the SLAs apply Thus a separate layer of grouping results based on the region ID must be applied The main aspect mentioned by all the providers is that the resources need to be reachable This is not completely determined as to what it means could be based on pinging for example ssh connection etc In our implementation this determination of connectivity will be decided via a separate class that can be altered or adapted
77. er YCSB wiki Getting Started Etics Home Page Online Available http etics archive web cern ch etics archive index htm Bonfire Project Home Page Online Available http www bonfire project eu CloudSuite Home Page Online Available http parsa epfl ch cloudsuite cloudsuite html 10 Filebench Home Page Online Available http sourceforge net apps mediawiki filebench index php title Filebench 11 Filebench WML Online Available http sourceforge net apps mediawiki filebench index php title Workload Model Language 12 ARTIST Consortium Deliverable D7 1 Definition and extension of performance stereotypes ICCS NTUA and other partners March 2014 13 Eclipse Documentation for project import Online Available http help eclipse org juno index jsp topic 2Forg eclipse platform doc user 2Ftasks 2Ftasks importproject htm 14 JCloud Documentation Online Available http jclouds apache org documentation devguides using eclipse 15 MongoDB Installation instructions Online Available http docs mongodb org manual installation 16 MongoVUE Desktop application Online Available http www mongovue com 17 Cloud4SOA Home Page Online Available http www cloud4soa eu 18 W3 Mathematical Modelling Language MathML Online Available http www w3 org TR MathML3 19 T Eilam M Kalantar A Konstantinou G Pacifici J Pershing and Agrawal Managing
78. erformance Version v0 6 Final Date 31 03 2015 analysis framework Google App Engine profile This profile has the same objective than Amazon EC2 and Windows Azure s but it is focused on Google App Engine specification needs The main difference is that taking into account that GAE is a PaaS profile it imports and makes use of PaaS stereotypes instead of laaS ones Next is described a set of supporting profiles These profiles have been created in order to respond to the representation needs of the project at this stage For now the set of supporting profiles is composed of Pricing Availability Security and Benchmark profile but it will be possible to extended it by adding other profiles in a quite simple way in case new requirements arise Furthermore they are independent of the CloudML ARTIST main profile thus can be individually used e g by other approaches Pricing profile Included in supporting profiles category this profile can be applied to any cloud provider to model pricing related aspects Availability profile Profile that permits to model cloud provider availability related aspects as these are expressed in the SLAs The stereotypes of this profile can be applied on different service elements e g ServiceOfferings in order to independently describe different SLAs that may apply to different types of services e g Compute SLA Storage SLA etc Security profile This profile is used to specify sec
79. es currencies and cost item This PriceType can be included to indicate the price of a variety of offerings such as storage and CPU or as a subtype in more complex types such as the networking costs Name Type Card Description onDemand PriceType 1 Pay per use pay as you go reserved ReservedPriceType 1 e g Amazon Reserved Instances spot SpotPriceType 1 Run instances as long as long as their bid exceeds the spot price PriceType PriceType parameters are shown in the following table Name Type Card Description price Float 1 Price of the offering currency String 1 Currency on which it is established the price perUnit Units 1 Price unit whose possible values can be selected from Units enumeration data type ReservedPriceType This type is related to the cost of reserved instances which gives customer the option to make a low one time payment for each instance wants to reserve and in turn receive a significant discount on the hourly charge for that instance Name Type Card Description Project Title ARTIST Contract No FP7317859 www artist project eu Page 120 of 129 D7 2 2 Cloud services modelling and performance analysis framework Version v1 0 Final Date 15 09 2014 price PriceType 1 Cost of the reserved price durationInDays Integer 1 Duration of reserved price SpotPriceType This type is related to
80. es representing paas iaas and saas providers extends from this stereotype and includes if necessary some features specific to that type of provider Next are described the properties contained in the stereotype MM description of the serviceDescription String High level type of service It serviceType ServiceTypes could be iaas paas or saas service offered service Scalability related features that can be applied to a scalabilityType ScalabilityType service l Service Level Agreement performanceSLA PerformanceSLAMetric related to performance Project Title ARTIST Contract No FP7 317859 www artist project eu Page 87 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework Service Level Agreement availabilitySLA Integer 0 1 related to availability offered by the service Data types Apart from this set of stereotypes described previously there is also a set of high level data types that will be used extensively in CloudMI Artist whose main purpose is to specify lists of common cloud world related values that can be reused when creating models from CloudML Artist Next a view of the package containing these high level data types Project Title ARTIST Contract No FP7 317859 www artist project eu Page 88 of 129 D7 2 3 Cloud services modelling and performance analysis framework CoreDataTypes Enumeration PublicPrivate
81. ew and Result 2010 30 B F Cooper A Silberstein E Tam R Ramakrishnan and R Sears Benchmarking Cloud Serving Systems with YCSB 31 Papyrus User s Guide Online Available http wiki eclipse org Papyrus User Guide 32 Eclipse Indigo Online Available http www eclipse org downloads packages eclipse modeling tools indigosr2 33 Amazon Amazon S3 Service Level Agreement Online Available http aws amazon com s3 sla 34 Yahoo Home Page Online Available https github com brianfrankcooper YCSB wiki 35 Aview from Berkeley Online Available http view eecs berkeley edu wiki Dwarfs 36 CloudSuite Data Caching Home Page Online Available http parsa epfl ch cloudsuite memcached html 37 Enterprise Integration Patterns Home Page Online Available http www eaipatterns com 38 Cloud Vertical costs comparison and characteristics Online Available https cloudvertical com cloud costs cloud_costs index 39 Evangelinou Galante Kousiouris Giammatteo and others Experimenting with application based benchmarks on different Cloud providers via a multi cloud execution and modeling framework Springer 2014 Project Title ARTIST Contract No FP7 317859 www artist project eu Page 80 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework Appendix A CloudML Artist core profile This section intro
82. form PerformanceTime ms S Amazon ec2 user DaCapoSuite Avrora P entos 18199 2 Openstack root DaCapoSuite Avrora Cent 6906 3 i LocalvM i root DaCaposuite Avrora CentOS i 7630 eee ET MGE SEG j i EE ENS Openstack do DaCapofukte Tomcat os EE 6 iLocalvM root i DaCapoSuite Tomcat Centos 18905 Figure 17 Example of Benchmark database view 4 2 2 4 1 Database REST interface The results obtained by the execution will be visible also via web in such sense a REST interface using FLASK has been developed FLASK is a micro framework for Python based applications In particular it is possible to distinguish the following absolute paths e http ipaddress port benchmarking ui will return the menu page e http ipaddress port benchmarking ui lt tool gt will return the entire result set for the tool http flask pocoo org Project Title ARTIST Contract No FP7 317859 www artist project eu Page 44 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework e http ipaddress port benchmarking rest results lt tool gt will return data in json format As an alternative it is possible to query the db directly from the browser http localhost 5000 benchmarking ui dacapo InstanceType m1 small amp amp CloudProvider Am azon Such path will return the performance data for DaCapo as case of Amazon instance m1 small http 127 der amaz
83. framework is that it is extensible it supports easy definition of new workloads in addition to making it easy to benchmark new systems There are two ways to define workloads e Adjust parameters to an existing workload via properties file e Define a new kind of workload by writing Java code In terms of metrics YCSB focus is related to performance and scale out we can consider two different benchmark tiers e Tier 1 Performance o Increase offered throughput until saturation o Measure resulting latency throughput curve e Tier 2 Scalability o Scaleup Increase hardware data size and workload proportionally o Measure latency should be constant o Elastic speed up Run workload against N servers o Measure timeseries of latencies The ARTIST Repository contains the version 0 1 4 of YCSB and the related test results it has been possible to test such version both on Windows and Linux Oss Figure 20 shows an example of YCSB output against a Cassandra database Project Title ARTIST Contract No FP7 317859 www artist project eu Page 46 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework p Wa X DN ex d n 9 e d Pom E Command line db com yahoo ycsD db CassandralclientlO p hosts l32 67 105 254 F workloads worrloada run OVERALL Ru nlime ms 63539 0 OVERALL Throughput ops sec 1872 2350254231089 INSERT Operations 100000
84. g tools in the context of ARTIST project More information about generation and maintenance of packages for benchmarking tools are available in Section 5 5 2 Installation instructions Installation of benchmarking tools is highly simplified by the fact that these tools have been re packaged to meet format of installation packages of most common platforms For example in the following we describe installation procedure for a CentOS machine 1 accessto target machine previously provisioned by the cloud provider 2 add ARTIST Repository to machine software repositories cd etc yum repos d wget http etics res eng it repository pm registered repomd name artist benchmarking etics registered build by name repo Project Title ARTIST Contract No FP7 317859 www artist project eu Page 56 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework Now we will be able to use the typical yum commands e g search or install to manage ARTIST packages e g Figure 31 All software released by ARTIST will be installed in the prefix opt artist qal 64 Figure 31 Installation procedure of benchmarking tools For platforms not directly supported Redhat and Debian based it is still possible to use the tar gz package to install software 5 3 User Manual Concerning the Benchmarking Suite e it is possible to refer to this document and its future updates for installat
85. hanisms for the migrated application In the technical feasibility analysis this information will be considered in order to calculate the complexity of supporting such mechanisms by developing a dedicated new component or using a service provided by the target platform The proposed possible authorization mechanisms are SAML OpenID others In the Business Feasibility Analysis the target platform information will be used to determine the most suitable business model to check if the selected platform if it is already decided by the user accomplish the business related migration goals or to propose the most adequate target platform that supports the license model and the business model that the user is willing to offer to their customers The envisioned required information from the target platform to the Business Feasibility Analysis is Project Title ARTIST Contract No FP7 317859 www artist project eu Page 20 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework e Billing information of the target platform The Business Feasibility Analysis will make some calculations to provide estimations of the Cost Benefit Analysis In order to perform this analysis information about the cost of the service e Pricing model supported by the target platform The user expresses her needs and wishes in the maturity analysis and the target platform should be able to support them In the co
86. hey may be abstracted and be used for all cases For a calculation to be performed the user must enter the month of interest in date format The PingReachabilityRuler is a class hiding the implemen tation of the reachability determination which is implemented in the PingThread class It was Project Title ARTIST Contract No FP7 317859 www artist project eu Page 60 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework designed in this way so that the way reachability is determined can be easily replaced in the future in case of a more appropriate method than pinging is considered The StreamGobbler class is necessary to consume the output of the system command thread to execute the ping lt lt Java Class gt gt lt lt Java Class gt gt 9 TriggerAvailabilityCalculation G RunaAvailabilityAuditingDaemon eu artist cloud auditors arti i TriggerAvailabiityCalculation GF RunA vailabilityAudtingDaemon main String void main String void createPropertiesFile void lt lt Java Class lt lt Java Class gt gt 9 AbstractedAvailabilityCalculator 9 AvailabilityAuditor eu artist cloud auditors eu artist cloud auditors AbstractedA vailabiltyCalculator A providerName String calculateAvailability String int double 4 serviceName String amp calculateCloudSleuthA vailability String double 4 user String dbhost String 4
87. hird party benchmarking tools being part of the Benchmarking Suite distribution varies from tool to tool The most common technology used is by far the C C programming language preferred because of its low access to machine resources and the small memory footprint Also Java and Python technologies are largely used 4 2 4 Third party benchmarking tools We have implemented the repackaging of a selected set of third party benchmarking tools delivered through the ARTIST Repository We will complete the overall architecture integration between benchmarking controller and GUI during the third year of ARTIST 4 2 4 1 YCSB YCSB 4 is one of the most popular database benchmarking tool Basically it is an open source workload generator and can be considered a standard benchmark able to evaluate different systems on common workload Project Title ARTIST Contract No FP7 317859 www artist project eu Page 45 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework Command line parameters DB to use Target throughput s Number of threads Workload parameter file YCSB dlient RW mix Record spe Data set Extensible define new workloads Figure 19 YCSB Architecture Cloud DB In a nutshell it is a Java framework used for generating the data to be loaded to the database and the operations which make up the workload A key feature of the YCSB
88. iagram 6 2 2 1 Key assumptions and implementation considerations Due to the ambiguous nature of legal contracts or the inability to define every technical detail in the latter during development we have made the following assumptions considerations During each loop we check every time template IDs that are valid according to the provider preconditions and are admissible for the SLA This happens because the valid IDs may vary over time due to user actions e g starting stopping a service instance affects the number of run ning VMs per template which is a basic precondition for an SLA to apply Only for these valid Project Title ARTIST Contract No FP7 317859 www artist project eu Page 61 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework template IDs we log the availability sample of that specific period in order to keep only sensi ble and usable information and reduce the amount of data to be stored and later processed We assume an arbitrary interval of 5 minutes for which consecutive timestamps of the logs are considered in the same interval This practically means that if we find that 2 consecutive data base entries have difference in timestamps larger than 5 minutes we assume that during this time the monitoring was stopped or other actions have deemed the SLA inapplicable for the specific template ID Thus the second sample is not counted in any previously id
89. iate data structure java objects in order to be inserted in the included placeholders of a cloud provider profile which is retrieved from the ARTIST repository After having the updated cloud provider profile Model Updater restores it in the ARTIST repository The Model Updater process appears in Figure 38 Finally it is worth mentioning that this prototype facilitates the aforementioned procedures by providing an automated way and relieving the user from the manual population of the profiles avoiding the possibility of inserting incorrect score values Model Updating process Benchmarking Tool Benchmark Results DB MySQL Queries CloudProvider InstanceT ype amazon milarge mi medum m1 5mal timro UML Model Elements ARTIST repository Transform to AYG Runtime_ms 9025 125 Transform 2880 869 52173915 to 8199 6 2646 53503 18471337 2102 2741935483873 Java Objects 17847 0525 197969 14285714287 Figure 38 Model Updater process Project Title ARTIST Contract No FP7 317859 www artist project eu Page 73 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework 8 2 Technical description Model Updater prototype consists of three components the DB Agent which is responsible for establishing database connection and executing MySQL queries the Benchmark results data model which provides the appropriate data structure
90. ices or client libraries both could be seen as an API we basically need 3 things an Interface Configuration and Security Stereotype SoftwareService authenticationMethods AuthenticationMethod host String 0 1 interface configuration IL Ve H Stereotype Stereotype Stereotype Property Configuration Interface I 0 1 property lea it 1 key String specification String 1 value String 1 1 eme dees UI language String Figure 46 SoftwareService Stereotype This stereotype allows modelling the software services a SaaS provider can offer It extends from Service stereotype defined at a higher level whose description can be found on the beginning of this appendix It contains a name and optionally a host URI pointing towards the actual service in client libraries this would not be necessary In the following table the properties contained in the stereotype are described Wee mm eem authenticationMethods AuthenticationMethod Type A needed to access the service String Host where is allocated the service category SoftwareServiceCategory A category describing the functionality provided by the service Project Title ARTIST Contract No FP7317859 www artist project eu Page 102 of 129 D7 2 2 Cloud services modelling and performance Version v1 0 Final Date 15 09 2014 analysis framework Every SoftwareService supports a number of
91. ider can offer CommonFeatures CommonFeatures pricingPlan PricingPlans certification Certifications scaleup Boolean 1 scaleout Boolean 1 4 monitoring HighLevelEvaluation 1 support HighLevelEvaluation 1 4 api HighLevelEvaluation 1 freeTier Boolean 1 typeAppOriented ApplicationTypes scope PublicPrivate 1 availabilityZones AvailabilityZones gpuInstances Boolean 1 loadBalancing Boolean 1 supportsOCCI Boolean 1 supportsMultitenancy Boolean 1 supportsFederation Boolean 1 usageType UsageTypes This stereotype offers the possibility to specify common features that can be applied in general to any cloud provider Apart from this stereotype specified at a high level there are also equivalent stereotypes at iaas paas and saas profiles see in next sections extending from this one and allowing setting specific properties at those levels Next are described the properties contained in the stereotype m FF Project Title ARTIST Contract No FP7 317859 www artist project eu Page 84 of 129 D7 2 3 Cloud services modelling and performance analysis framework pricingPlan certification scaleup scaleout monitoring support Project Title ARTIST PricingPlans Certifications Boolean Boolean HighLevelEvaluation HighLevelEvaluation Page 85 of 12 Version 9 v0 6 Final Date 31 03 2015 pricing pl
92. inition 7 4 Licensing information 3ALib is released under the Apache License v2 0 7 5 Download 3ALib is available for download from the Artist Repository https github com artist project ARTIST Tooling tree master migration target environment specification ArtistAuditor Project Title ARTIST Contract No FP7 317859 www artist project eu Page 72 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework 8 Model updater Implementation 8 1 Functional description Model Updater prototype is a supporting tool which is used in order to populate and update the performance profiles of CloudML Q ARTIST framework stored in ARTIST repository with the respective values from the benchmarking process As already mentioned the overall benchmark measurement results are stored in the internal WP7 Raw Data DB in order to avoid extra overloading the uml profiles and keep the model as simple as possible in the ARTIST repository The main purpose of the Model Updater is to run periodically and refresh the profiles with the latest average performance scores by keeping the CloudML ARTIST updated at regular intervals In order to do this the Model Updater is responsible for establishing a database connection executing specific MySQL queries in order to calculate the average scores for each generic benchmark category The previous performance information is transformed in an appropr
93. ion log samples The steps after installation are optional since in Mongo if the specific db and collection are not created beforehand in the database they will be once an initial attempt is made to write to them Thus the first time one runs the 3alib tool the relevant collection and db will be auto matically created Since it is a NoSQL based approach no further schema or structure needs to be defined The information is inserted in a row based logic throughout 3alib The location of the DB along with the credentials needs to be provided to the configuration file of 3alib An interesting GUI tool that we have used for viewing the contents of the database and can be used also to perform various management operations is MongoVUE 16 An indicative screen shot is provided be low Figure 35 The id column includes also the timestamp of inclusion which is also the timestamp used for the comparison of time differences In case we need to minimize logged data we can also reduce the number of columns stored Ideally we only need a limited number of the available information of each line timestamped id Status template ID availability re gion and resource ID Project Title ARTIST Contract No FP7 317859 www artist project eu Page 68 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework v xX Refresh Table View Text View 0 100 Tree View 67 Documents
94. ion and configuration e to properly run a benchmark using the third party tools users can refer directly to the user s documentation of the tools usually available on line on the tools homepage 5 4 Licensing information As far as the Benchmarking Suite is concerned it has been released under Apache 2 0 The third party benchmarking tools repackaged in a standard format and stored in the ARTIST Repository are open source and released under licenses that allow repackaging and redistribution Next table lists licenses for each third party tool redistributed in the Benchmarking Suite release Last column also lists if any the dependencies and correspondent licenses that the tool uses at run time These dependencies are not distributed through the Benchmarking Suite but downloaded from other software repositories and or distribution locations Project Title ARTIST Contract No FP7 317859 www artist project eu Page 57 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework Table 2 Benchmarking Suite Licenses Tool License Dependencies YCSB Apache 2 0 MySQL GPL 2 0 Cassandra Apache 2 0 MongoDB AGPL 3 0 un Poo B 8 MEM CloudSuite Data CloudSuite 2 0 Memcached BSD Caching US 5 5 Download Benchmarking Suite is available on the ARTIST Repository and accessible following instructions presented in section 5 2 An additional access point for browsing and
95. is summarized in the Table 6 Project Title ARTIST Contract No FP7 317859 www artist project eu Page 69 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework Table 6 3ALib accepted provider names The configuration file can be edited via any standard editor or created via code as in the fol lowing example public static void createPropertiesFile Properties prop new Properties OutputStream output null try output new FileOutputStream 3alibconfig properties set the properties value change them also in properties file can be deleted from here prop setProperty databaselP 192 168 56 101 prop setProperty user YOUR USER prop setProperty APIkey YOUR API KEY prop setProperty ProviderName aws prop setProperty ServiceName ec2 prop setProperty DBuser YOUR DB USER prop setProperty DBKey YOUR DB KEY save properties to project root folder prop store output null catch IOException io io printStackTrace finally if output null Project Title ARTIST Contract No FP7 317859 www artist project eu Page 70 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework try output close catch IOException e e printStackTrace 7 3 2 Executions 3ALib can be executed either as an individual ja
96. itecture is shown in Figure 26 DRIVERS input Quinn Adapter Adapter f jm OUTPUT RESULTS CEP Engine A X ei Controller S ng Fides er Log Piles or CN Runtime Samples Runtime Samples fas Performance Monitor Figure 26 FINCoS framework architecture It is possible to distinguish five different components e Driver simulates external source of event its configuration allows to specify the load to submit to the system to test and the typology of workload synthetic generated by user directives or based on external files 3 https code google com p fincos Project Title ARTIST Contract No FP7 317859 www artist project eu Page 51 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework e Sink receives output events resulting from the queries running on CEP engines and stores results in log files e Controller interface between framework and user responsible for the environment configuration phases e Adapters converts events from the framework into a target system compliant format the tool communicates with CEP engines directly via customized plug and play adapters or via JMS messages e Performance Monitor collects performance metrics both real time and offline In the following an example of performance results related to a simple test scenario awe be Se ee S aged i
97. ition which is described below contains the general information of SLA definition which is specialized in PaaSProviderAbstractedDefinition laaSProviderAbstract edDefinition and StorageProviderAbstractedDefinition subelements me JS Ja qeemm Compensation DiscountType 5 Compensation that customers claim after proofing the violation QuantumOfTimelnMi Integer 1 Quantum of nutes downtime period CalculationCyclelnDay Integer 1 Calculation cycle of a S month StatusAcknowledgem StateDiscovery 1 Representation of ent status availability GuaranteedPercentag 1 Availability e guaranteed percentage provided by SLAs ServiceName String 1 Unique name of service Next is described the only property contained in the PaaSProviderAbstractedDefinition stereo type which indicates in an abstracted way the necessary preconditions must typically apply for an SLA to be applicable regarding PaaS providers Project Title ARTIST Contract No FP7317859 www artist project eu Page 107 of 129 D7 2 2 Cloud services modelling and performance Version v1 0 Final Date 15 09 2014 analysis framework fame mm remm Preconditions PaaSOptions necessary preconditions must typically apply for an SLA to be applicable laaSProviderAbstractedDefinition field includes parameters that result from comparing differ ent laaS provider s SLA definitions and identifying common concepts and differences Infor mation regarding the fields
98. ively we use mongo 2 4 java client for interaction with the DB backend gson 2 2 4 for creat ing gson objects for db storage guava 16 0 and guice 3 0 6 2 4 Availability experiments on Amazon AWS EC2 In order to test the implementation of the tool we executed a 1 5 month experiment against an Amazon EC2 deployed service consisting of 2 virtual machines running on the North Virginia Datacenter The two VMs were configured to be launched in different availability zones us east 1c and us east 1b in order to comply with the Amazon SLA prerequisites and they were continuously running The experiment included also other VMs consideration in the Oregon Datacenter that were used for other tasks and were also considered that had more dynamic deployment aspects were being started and stopped at arbitrary times The experiment was executed from November 2014 to 15 of December 2014 The results of the experiment appear in Table 3 From these it is evident that there is significant difference between the availability calculations based on the specific provider s SLA and the generic formula used by Cloudsleuth It is evident that with only one extra unavailable sample in the Oregon case the Cloudsleuth defined availability would indeed violate the 99 95 boundary and thus appear to have an SLA violation However because the unavailable samples are not consecutive they are indeed isolated the minimal 1 minute interval of continuous unavailabi
99. java objects of performance scores and the Model parser which is responsible for the interaction with the UML profiles A detailed analysis for each component is presented in the following subsections DB Agent DB Agent is responsible for accessing the database and performing the required MySQL queries in order to obtain the average values coming from benchmark results So for every instance type of each provider there is a set of benchmark tests performed each of which comes with a set of measured properties This component connects to the database performs the queries and stores the results in memory following the data structure provided by the results data model component Benchmark results data model This component provides the data structure used for data exchange between the dbAgent and the model parser The abstract class BenchmarkResults is defined as shown in Figure 39 public abstract class BenchmarkResults protected String provider protected String instanceType protected ArrayList lt String gt results public abstract List lt String gt getPropertyNames public abstract String getBenchmarkType public boolean setInstanceType String instanceType public boolean setProvider String provider public void addResult Double result _ public String getProvider Figure 39 Abstract class BenchmarkResults Other less important methods are also included A set of classes extend this abstract class and c
100. lated to the execution is in plan as future development 4 2 2 4 Benchmark Database The results obtained by the execution will be parsed first and transferred back then in order to be processed and included in the CloudML model descriptions In addition a mysql raw database schema has been created and provided in case the results from the benchmark tests needed to be stored locally The database structure is depicted in Figure 16 Actually the GUI and Controller components are not integrated however the benchmarking controller the main backend component can also be used standalone to perform measurements on the various application types An example of mysql table containing benchmark results is depicted in Figure 17 Project Title ARTIST Contract No FP7 317859 www artist project eu Page 43 of 129 D7 2 3 Cloud services modelling and performance analysis framework v B YCsB v Columns Version vO 6 Final Date 31 03 2015 v ES CloudSuiteDataCaching v Lei Columns CloudProvider CloudProvider D gt User a i User Tool di Tool Workload v Tables Workload Platform gt SS CloudSuiteDataCaching em Platform Runtime ms gt Mi DaCapo TimeDiff Throughput ops sec FS Dwarfs Rps UPDATE Operations FS Filebench Requests UPDATE Averagelatency ms FS YCSB Gets UPDATE MinLatency ms Sets UPDATE MaxLatency ms Hits UPDATE 95thPercentileLatency ms Misses UP
101. le ARTIST Contract No FP7 317859 www artist project eu Page 13 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework e Extension of 3ALib to support Google Compute Engine SLAs e A definition of a new metric for SLA strictness e Inclusion of SaaS instances for billing and monitoring e Integration of the provider models in the Cloud Target Selection process in WP9 Project Title ARTIST Contract No FP7 317859 www artist project eu Page 14 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework Introduction The main objective of Work Package 7 is to provide the instruments for modelling target cloud environments where migrated applications will be deployed Cloud models will be used in other work packages as inputs for processes such as the business and technical feasibility analysis Work Package 5 forward engineering Work Package 9 and migrated application testing and validation Work Package 11 Cloud models must capture all the characteristics and data of cloud providers needed by these other processes in the project Requirements concerning the type and the format of information needed in cloud models have been investigated and elicited in project deliverable D7 2 PaaS laaS Meta modelling Requirements and SOTA 1 thanks to a close collaboration with the other technical work packages Followi
102. licatinTy 1 Types of replication of data pes the provider can offer Values can be established by using StorageAccountReplicatinT ypes entity laaSNetworkService Project Title ARTIST Contract No FP7 317859 www artist project eu Page 93 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework Stereotypes laaSNetworkService max bandwidth in Integer 0 1 max bandwidth out Integer 0 1 This stereotype allows modelling network services by an laaS provider It extends also from Service stereotype Next are described the properties contained in the stereotype max bandwidth in Integer 0 1 Maximun input bandwidth capacity offered by the provider in Mbps max bandwidth out Integer 0 1 Maximun output bandwidth Capacity offered by the provider in Mbps laaSinstanceType Enumeration Enumeration InstanceCategories ArchitectureTvpe General Purpose Memory Optimized Arch 64 B it Storage Optimized Arch 32 Bit IaaSInstanceType OSs instanceCategory InstanceCategories 1 E c Centos Enumeration BEER region AvailabilityZones 1 Keen RedHatEnterpriseLinux AvailabilityZones operatingSystem OSs 1 Debian East US GracleEnterpriseLinux virtualCares Integer 1 North Central US WindowsServer Enumeration memory Real 1 teal SUSELinuxEnter prise localDisk Integer 1 So
103. licationTypes Type o applications to which it is oriented the provider Values given by ApplicationTypes enumeration data type PublicPrivate 1 Indicates if the provider is public or private Geographical zones where availabilityZones AvailabilityZones 9 WW supportsOCCl Boolean ee Project Title ARTIST Contract No FP7 317859 www artist project eu the provider offers its infrastructure Indicates whether the provider offers gpu instances or not Indicates whether the provider offers load balancing capabilities or not Indicates whether the provider supports occi Open Cloud Computing Interfac or not Indicates whether the provider supports Page 86 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework supportsFederation Boolean Indicates whether provider supports federation or not usageType UsageTypes Usage types offered by the provider Possible values can be selected by using UsageTypes enumeration data type Service Stereotypes Service serviceDescription String 0 1 SeryiceType ServiceTypes scalabilityType ScalabilityType 0 1 performanceSLA PerformanceSLAMIetric 0 1 availabilib SLA Integer 0 1 serviceHMame String 1 This stereotype is used to describe the main information related to a service that can be offered by a cloud provider Every service included in the profil
104. lity is not met thus the provider defined availability is at 100 Table 3 Availability Experiment Statistics Month Samples Unavailable AWS SLA CloudSleuth Defined Defined Availability Availability 2 Testing N Virginia November VMs AWS 2014 W all 99 9968 Arbitrary 99 9547 VMs 2 Testing N Virginia December VMs AWS 2014 EC2 Project Title ARTIST Contract No FP7 317859 www artist project eu Page 63 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework As an indication if the SLA defined availability was between 99 and 99 94 our experiment for the N Virginia part would cost 114 S if it was below 99 it would cost 88 95 whereas now the estimated cost is about 127 S Sampling Details and Statistics The probability of the samples in terms of interarrival times obtained from subtracting the server timestamped data as they were inserted in MongoDB at the client side appears in Figure 34 This delay includes networking delay from the client location Athens GR to the data center N Virginia US Amazon API delay for acquiring the server list and pinging delay for the two VMs pinging delay for the external known locations that were used to ensure client connectivity and Mongo insertion delay Potential loss of this client side connectivity is probably responsible for the extreme cases of sample interarrival delays e g in the range of
105. low making semi automatic deployments of the legacy application to the selected provider by applying M2T transformations 2 2 1 Meta model Structure Next figures describe how CloudmI Artist is structured as a series of interconnected UML profiles making possible to create models with great flexibility Project Title ARTIST Contract No FP7 317859 www artist project eu Page 25 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework Core Profile Common stereotypes and data types Figure 4 Core Profile Specific providers profiles can be extended Figure 5 Specific providers profiles Project Title ARTIST Contract No FP7 317859 www artist project eu Page 26 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework Pricing profile Availability profile gt Stereotypes and gt Stereotypes and data types related E data types related to pricing to pricing Security profile Benchmark profile gt Stereotypes and gt Stereotypes and data types related mm data types related H to security to benchmarks results Figure 6 Supporting profiles 2 2 2 UML Profiles Description As described in previous section CloudMI Artist is organized as a set of UML profiles with hierarchical relationships between them Next a brief description of each profile Cor
106. lysis framework 4 2 Temaer 36 4 2 1 Prototype AFchite Cl UNG vr 36 4 2 2 COMPONENTS DESCIIDTION BEE 37 4 2 3 Technical specication EE 45 4 2 4 Third party benchmarking tools 45 4 2 5 Benchmarking experiments and related performance model population 53 5 Benchmarking Tools Delivery and Usage 56 51 PONTA 56 5 2 Hastallation re ae 56 5 3 Ser Manavi GANGE 57 5 4 Breet ein ATION edda 57 5 5 Belle Ce e TEE 58 o lt SALID leet EE ECH EE 59 6 1 FUNCOMS DES CHIDE EE 59 6 2 TECHMICAID CSCUIDUIOM NE 59 6 2 1 Prototype greet ae 60 6 2 2 COMPONENTS DES EECHELEN 60 6 2 3 Technical SDE CIICALION RE 62 6 2 4 Availability experiments on Amazon AWS ECH 63 6 2 5 AEG ve 65 y Ee NEE Ee LEET 68 7 1 ie leg ue DEER 68 7 2 KSt ll ton Inseis 68 7 3 User Manual WEE 69 7 3 1 EM 69 7 3 2 ege E 71 7 4 Heensns Mom NN 72 7 5 OW OA EO P 72 8 Model updater lmplementation 73 8 1 Fund 73 8 2 Seiler Reeg ege EE 74 8 2 1 Tec SOC CINICAUION EE EE 75 9 Model Updater Delivery and Usage Luse 76 9 1 Packaceiniorma ON aeia E 76 Project Title ARTIST Contract No FP7 317859 www artist project eu Page 6 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework 9 2 HS EON INST UO se dd 76 9 3 CISC Ii et Ulli PE EE ERE ILU NE AT UT 76 9 4 licensing Nome 77 9 5 PN 77 10 Cove 78 Ree 79 Appendix A CloudML Q Artist core profile 81 lad S Sedat
107. m Access Memory E European Commission EPL Eclipse Public License ESB Enterprise Service Bus ETICS elnfrastructure for Testing Integration and Configuration of Software GAE Google App Engine GNU GNU s Not Unix GPL General Public License GUI Graphical User Interface I O Input Output laaS Infrastructure as a Service IDE Integrated Development Environment noSql Not Only SQL O Operating System PaaS Platform as a Service Project Title ARTIST Contract No FP7 317859 www artist project eu Page 11 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework PSM Platform Speficic Model s Project Title ARTIST Contract No FP7 317859 www artist project eu Page 12 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework Executive Summary This document is associated to four prototypes delivered in Work Package 7 The prototypes aim at providing a modelling and performance analysis framework to support the creation of cloud models and to measure performance or availability aspects of cloud services Implemen tation and usage instructions details are included in this document The first prototype is a meta model that will be used as baseline for creating models of cloud models Section 2 The meta model defines the concepts and relationships that describe the main capabilities of resource
108. meration type PaaS This section introduces a proposal for extension to CloudML ARTIST in order to incorporate Project Title ARTIST Contract No FP7 317859 www artist project eu Page 96 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework PaaS concepts into this meta model aligned with the laaS description already available in CloudML At this stage the PaaS extension built as an UML profile describes the main PaaS stereotypes and their relationships linking some of them to available CloudML stereotypes By the use of this profile in conjuction with the rest of stereotypes and entities defined at the higher level in CloudML Artist described previously it is possible to model the main characteristics of any PaaS provider such as Google App Engine for example In the elaboration of this PaaS extension we have also borrowed some ideas and concepts from Cloud4SOA Semantic Model for PaaS 8 Error No se encuentra el origen de la referencia depicts the suggested PaaS extension Key concept is the notion of CloudOffering which generalizes the one borrowed from PaaS domain 8 A CloudOffering is a package of services a CloudProvider offers to users for different purposes depending on the layer of the Cloud stack Project Title ARTIST Contract No FP7 317859 www artist project eu Page 97 of 129 D7 2 2 Cloud services modelling and performance analysis framework
109. migration model updater m JRE System Library java 7 oracle m Plug in Dependencies v src gt 83 eu artist migration model updater benchmarkres gt Bj eu artist migration model updater dbagent gt amp eu artist migration model updater modelparser gt Referenced Libraries 2 META INF Figure 40 Model Updater packages 9 2 Installation instructions Installation Instructions Install Java 7v or later if not already installed in your system Download the runnable jar file from the ARTIST public repository 9 3 User manual Execution Instructions Prerequisites Before executing the tool the users should make sure that they have 1 created a txt file for the name mappings between the database and the models The file must contain one section for provider names mapping and one section for instance types mapping Each section must have a title either providers or instances A txt file sample is shown in Figure 41 After each section s title the names are written first the name for the database and afterwards the name of the provider following a space character input txt 23 providers amazon amazonec2 profile umL azure windowsazure profile uml instances ml medium MlMediumInstance Al Al A2 A2 Figure 41 txt file sample Project Title ARTIST Contract No FP7 317859 www artist project eu Page 76 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 a
110. nalysis framework The two last lines stand for two different instance types of Microsoft Azure cloud platform It is important to point out that although these instance types have the same names for the database and the meta models they must be included in the input txt file or else they will be ignored during the model update Bottomline this file should include only the providers and the instances that are to be updated with the database values 2 found the local directory of the CloudML Q ARTIST metamodel 3 the ip address the username and the password for accessing the benchmark results database Execution For executing the runnable jar the user should open a command line and type java Jor runnable jar file i ip address u databaseusername p database password f lt input file with name mappings m CloudML Q ARTIST metamodel directory After the execution the new values should be stored in the model files 9 4 Licensing information Model Updater is offered under the Eclipse Public License EPL 9 5 Download Model Updater is available for download from the Artist Repository at https github com artist project ARTIST Tooling tree master migration target environment specification eu artist migration model updater Project Title ARTIST Contract No FP7 317859 www artist project eu Page 77 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 anal
111. nctional goals are implicitly given by the legacy software whose observable functional behaviour should not change during the migration These goals are evaluated before the deployment by generating test cases from the migrated model and after the deployment via end user based testing Non functional goals i e goals that are concerned with quality aspects of the software such as performance are collected from the user with the maturity assessment tool during the pre migration phase and stored in a so called goal model In general non functional goals are evaluated for a given workload by extracting the respective execution traces from the executed migrated model or from monitoring the deployed migrated software as depicted in Figure 2 Model level validation can be done before the actual deployment and thus enables to avoid costly deployments that will most probably not fulfil the migration goals set by the user This is done by simulating the behaviour of the migrated software based on the migrated models For estimating the expected non functional properties by model simulation as accurately as possible the cloud model will serve as valuable input regarding several characteristics of the underlying cloud platform e g hardware characteristics scaling strategies etc Therefore cloud models can be used to obtain more accurate measures for certain metrics e g execution time of a specific operation on a given platform by providing detail
112. ng the Model Driven Architecture MDA approach Work Package 7 aims at defining a meta model that specifies all the concepts and relationships of interest when modelling a cloud provider This meta model is called CloudML ARTIST since it is directly derived from a continuation of the work started in the REMICS project In particular the CloudML ARTIST meta model contains some concepts not covered by the original CloudML in such sense we have followed an iterative incremental approach and significantly improved CloudML in the second year of the project In particular e PaaS and SaaS offering the original CloudML only focuses on resources at laaS level e performance and monitoring aspects they are not taken in consideration in the CloudML meta model but needed in ARTIST to make decisions about the best target environment for the migration and the evaluation of the effectiveness of the migration e other aspects like pricing scalability availability that are of interest in ARTIST for the non technical evaluation of the migration process e g the business feasibility The second prototype released by Work Package 7 is a Benchmarking Suite with the main objective of standardizing and automating as much as possible the gathering and modelling of performance data of cloud resources The Benchmarking Suite is composed of four main components e a set of selected third party benchmarking tools that have been packaged and made available in th
113. nginterface for consistency purposes however they implement its methods based on each respective provider s logic and SLA They also use abstracted methods like the JcloudsEx ecutorClient used to retrieve information from the providers regarding the user s services and their status based on Apache Jclouds and the AbstractedAvailabilityLogger The latter is re sponsible for getting the detailed status report and storing it Furthermore it is responsible for hiding the differences variations in reported status messages from providers and including only a REACHABLE or UNREACHABLE status The storage part is based on a MongoDB database backend that may be created on the user s side or may be offered by a relevant service in a more exploitable scenario The purpose of using a NoSQL solution is that it is more scalable and can be used for parallel processing of the logs which is expected to be a time consuming process Further analytics queries can also be implemented based on advanced business intel ligence scenarios The final calculation of the SLA adherence levels is performed by the Ab stractedAvailabilityCalculator class which is abstracted since all providers follow a similar for mula for the calculation of availability but with a number of differentiated parameters such as the minimum continuous time interval for which a resource may be considered unavailable Thus provided that the methods in this class take these parameters as arguments t
114. ntext of the Business Feasibility Analysis the more convenient target platform in terms of supporting the business model required by the user will be selected or it will be checked if the target platform already selected by the user supports user s requirements For this the following information is envisioned to be required from the target platform o provided resources measurement models e time volume sessions requests users others o Supported billing rules i e flat rate cumulus rating by use by use minimum effective bandwidth pricing cumulus pricing responsive pricing others o Supported license models i e by subscription by use by transaction by value flat rage hybrid others 1 4 3 Usage of cloud models in the forward engineering process Several tasks in the forward engineering process provided by WP9 rely on the information provided by this deliverable In particular the knowledge about the functional and non functional properties of Cloud services is crucial for migrating legacy models produced by components of WP8 to Cloud based applications Here we would like to highlight three tasks that are the cornerstones of the modernization process Manual service selection The first basic step when migrating legacy models to the Cloud is to select manually the services that are desired for the operation of the system on the Platform Independent Modelling PIM level In this respect one may select generi
115. odeMetadata boolean i NN 6 addReachabiityinfo boolean NodeMetadata String LT mos Java Gass gt gt lavaCass ssdavaass gt 77 ena wa Class gt gt PingThread PingReachabilityRuler Lol cc 9 JCloudsExecutorClient Cu Auditinginterface AbstractedAvailabilityLogger eu artist cloud auditors i i eu artist cloud auditors eu artist cloud auditors eu artist d it S JCioudsExecutorCiient amp checkA pplicabilty Conditions String String String String String String void e AbstractedAvailabilityLogger Q getStatus String String logA vailability ArrayList lt String gt String String String void connectViaSSH String String String AbstractedGetServers String String String ComputeServiceContext n record String n input 1 String n input 2 String 4 finally reachable boolean 6 isFinally reachable boolean PingReachabiityRuler decideReachabiity String boolean setFinally_reachable boolean void S PnoThreadi lt lt Java interface al DecideReachabilityinterface run v i amp setRecord String void getRecord String 6 setinput_1 String void amp getinput_1 String setinput_2 String void amp getinput_2 String amp is Window s String boolean amp decideReachability String boolean isDecision boolean setDecision boolean void Figure 33 3ALib Class D
116. oject Title ARTIST Contract No FP7 317859 www artist project eu Page 23 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework 2 CloudML ARTIST Implementation Z 1 Functional Description In the context of the ARTIST project and following the analysis on the state of the art documented in the ARTIST Deliverable D7 2 1 it was decided to base our modelling framework on the CloudML 4 definition thus extending it to cover aspects specifically investigated in the project e g performance or better describe service offerings on different levels PaaS and SaaS A gap that was identified in the aforementioned document was the lack of adequate description frameworks for capturing performance characteristics of cloud services and resources For example for CPU resources typical descriptions like in CloudML include only number and frequency of CPU cores However this is far from sufficient for accurately describing the actual performance of a computing resource Furthermore fluctuation in the actual output of these services due to cloud environment issues e g noisy neighbour effect multi tenancy migration is a severe aspect that has begun to take notice in the cloud users Except for these measurable aspects a variety of services is available in the cloud market with different characteristics costs and purposes Thus an adequately rich description model should be in pla
117. ols provided by ARTIST are meant to be executed several times on all the infrastructures modelled also to capture variation in the performance values We believe that making it possible to manage execution and collection of data automatically saves a lot of time to users and produces better quality results 4 2 Technical Description 4 2 1 Prototype Architecture Benchmarking Suite architecture is depicted in Figure 7 Project Title ARTIST Contract No FP7 317859 www artist project eu Page 36 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework Execute Browse Data Publish Retrieve Target Environment I yum install artist ycsb mongodb artist ycsb mongodb all submit results Fetch tools Figure 7 Overall Benchmarking Suite Architecture At top level there are following components ARTIST Repository contains benchmarking tools used in the project to measure performance of cloud services It also contains results of tests in a standard format that can be consumed by other components Target Environment is the system under test In laaS offerings the target environment is a virtual machine running a given Operating System For PaaS the target environment is represented by services running in the cloud provider infrastructure and usable by applications deployed Benchmarking Controller is the main component of this architec
118. on ARTIST project promotes CloudML ARTIST as a standard for creating cloud models This may change depending on future needs but for the moment CloudML ARTIST is licensed under EPL 3 4 Download The CloudML ARTIST meta model is available as an Eclipse project in the source code repository hosted at GitHub The access to the repository is at the moment of writing restricted to project members only and to the European Commission The repository is reachable at the following link https github com artist project ARTIST tree master source Modelling Cloud9620Provider9620Meta model CloudMI 40Artist eu artist migration tes cloudml 40artist https github com Project Title ARTIST Contract No FP7 317859 www artist project eu Page 35 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework 4 Benchmarking Tools Implementation 4 1 Functional Description One of the extensions ARTIST contributes to the CloudML ARTIST meta model is the possibility of modelling performance aspects of services offered by cloud providers Along with the extensions defined in the meta model in order to ease the collection of performance data Work Package 7 will provide also a tool to configure execute tests and collect results based on a set of third party benchmarking tools in order to evaluate performance values that will be added to the cloud provider models The Benchmarking Suite
119. on x 127 0 0 1 5 arking 3capo instanceType all amp amp rov 3zon ve Fa Pidvisitatiy SSHtoAmazonEC2 W ovirt Engine User Por DMSF ARTIST R amp D AREA PERSONALE amp Adempiere python2 7 Show 10 lt entries Search Date CloudProvider InstanceType Workload Platform PerformanceTime ms 2014 04 09 11 59 59 Amazon ml small Avrora RedHat 48634 2014 04 09 12 10 06 Amazon ml small Avrora RedHat 44488 2014 04 09 12 14 12 Amazon ml small Avrora RedHat 43422 2014 04 09 15 26 17 Amazon ml small Avrora RedHat 50679 2014 04 10 12 01 01 Amazon ml small Avrora RedHat 46633 DaCapo Suite Filebench 2014 04 11 15 47 57 Amazon ml small Avrora RedHat 42405 Kies 2014 04 11 15 53 32 Amazon m1 small Avrora RedHat 46019 2014 04 11 16 01 31 Amazon ml small Avrora RedHat 46679 2014 04 11 16 07 21 Amazon ml small Avrora RedHat 46645 2014 04 11 17 40 19 Amazon mi small Fop RedHat 21449 Showing 1 to 10 of 742 entries Previous 1 2 3 4 5 ws 7 Next Figure 18 Example of REST interface for the database 4 2 3 Technical Specification The version of the Benchmarking Suite delivered is a set of selected third party benchmarking tools packaged and delivered thorough the ARTIST Repository The ARTIST Repository is compliant with the two most common standards for software repositories in the Linux environment Apt and Yum Packages are available in following formats deb rpm and tar gz The technologies used in the t
120. oncretize the abstract methods in such way so that 1 getPropertyNames returns the names of the properties for each benchmark test type exactly as these names appear on the Benchmark profile 2 getBenchmarkType returns the stereotype corresponding to this specific benchmark Two important methods for information exchange between the database and the models are the setinstanceType and setProvider methods These two methods get the string value as it appears in the database and sets the corresponding Java class attribute with the string value as it appears in the model In other words these methods actually perform a transformation Project Title ARTIST Contract No FP7 317859 www artist project eu Page 74 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework between the database and the model definitions Model Parser This component is designed in order to undertake the task of interacting with the UML profiles of CloudML ARTIST More precisely it provides the methods for accessing the corresponding models detecting the model elements and stereotypes which are to be updated and finally changing and storing the values of properties following the updates coming from the database The whole task is about updating the CloudML ARTIST metamodels from the point of view of benchmark profile applied on the providers profiles So upon receiving the requests from the dbagent
121. order to periodically refresh the average performance values that are included in the provider models In order to keep the latter lightweight it was decided that the raw data extracted from the measurements should be stored in an internal WP7 DB while their averages would be included in the actual profiles Thus this prototype is responsible for e calculating the average values from the raw data DB e retrieving the provider models from the repository updating their values and restoring them in the latter so that they are available to the overall ARTIST tool chain with the most up to date information 1 1 About this deliverable This document is delivered along with software prototypes released in Work Package 7 at the end of the third year It describes the main functional and technical aspects of the three prototypes released a the ClooudML ARTIST meta model b the Benchmarking Suite c 3alib library 1 2 Document structure The remainder of this document is structured as follow e Section 2 and 3 explain respectively the implementation of CloudML ARTIST and how it fits with the other tools and processes of ARTIST Furthermore the information is given about related delivered packages in terms of installation instructions licensing and download locations e Section 4 and 5 describe respectively the Benchmarking Suite tools implementation and usage e Section 6 and 7 describe respectively the 3Alib library implementation and u
122. ories General Purpose Memory Optimized Storage Optimized Compute Optimized Enumeration NetworkPerformancelype Enumeration StorageAccountReplicatin Types Geo Redundant Read Access Geo Redundant Locally Redundant www artist project eu D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework laaSProcessingService Stereotype Enumeration IaaSProcessingService OSs ArchitectureTvpe 4 numProcessors Integer 1 CentOS memory Integer 1 RedHatEnterpriseLinux Arch 64 Bit platform OSs 1 Debian processorArchitecture Architecture Types 1 Arch_32_Bit OracleEnterpriseLinux WindowsServer SUSELinuxEnterprise openSUSELinux SUSEEnterpriseLinux This stereotype allows modelling processing capabilities offered by an laaS provider It extends from Service stereotype described in the beginning of this appendix Next are described the properties contained in the stereotype LIEN ee numProcessors Integer Maximum number processors the udi can offer memory Integer Maximum amount of RAM the provider can offer GB platform Relation of Operating Systems supported by the provider Represented by the Enumeration type OSs on the right side of the picture processorArchitecture ArchitectureTypes Property through which it is specified if the provider supports 64 bit architecture 32 bit architecture or both Use
123. ork in order to assist with provisioning decisions for cloud users During the execution process we utilized workloads from DaCapo benchmarking Suite YCSB framework and Filebench The selected workloads from each test were running on instances in three different cloud environments Amazon EC2 Microsoft Azure and Flexiant In Flexiant s case cloud resources were provided by MODAClouds in order to execute benchmarking experiments as part of ARTIST and MODAClouds eu projects collaboration For each cloud provider different types of VM instances were selected in order to examine as much cloud resources as possible and give a complete view of the performance of the provided cloud offerings Information regarding the selected VM instance characteristics is presented in Table 1 The execution of the tests took place at specific hours at different time intervals during a period of eight months from July 2014 to 28 of February 2015 and the average values were extracted for each case Moreover the different time zones of the three respective regions were taken into consideration so that the peak hours were the same in each zone In Amazon EC2 case the virtual machines were running in North Virginia datacenter while for Microsoft Azure and Flexiant in Ireland and United Kingdom respectively Table 1 VM instance characteristics Cloud Provider Amazon EC2 Flexiant After completing the benchmarking process the results are retrieved from
124. ory loads The DaCapo suite consists of the following benchmarks e AVRORA simulates a number of programs running on a grid of AVR micro controllers e BATIK produces a number of Scalable Vector Graphics SVG images based on the unit tests in Apache Batik e ECLIPSE executes jdt performance tests for the Eclipse IDE e FOP parses formats XSL FO file and generates a PDF file e H2 executes a JDBC benchmark using a number of transactions against a banking model application Project Title ARTIST Contract No FP7 317859 www artist project eu Page 50 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework e JYTHON interprets pybench Python benchmark e LUINDEX uses Lucene to indexes a set of documents e LUSEARCH uses Lucene to search of keywords over a data corpus e PMD analyzes a set of Java classes for a range of source code problems e SUNFLOW renders a set of images using ray tracing e TOMCAT runs a set of queries against a Tomcat server retrieving and verifying the resulting webpages e TRADEBEANS runs the Daytrader benchmark via Java Beans to a GERONIMO back end e TRADESOAP runs the Daytrader benchmark via SOAP to a GERONIMO backend e XALAN transforms XML documents into HTML ones 4 2 4 7 FINCoS FINCoS framework is a java based set of benchmarking tools for load generation and performance measurement of Event Processing systems management Its arch
125. ovider models defined in ClooudML ARTIST The indicative formula for the case of equal importance to all parameters appears in Equation 3 S t 1 s q s p x where s normalization factor for the continuous variables so that s q 0 1 and s p 0 I t 10 11 xe 0 1 Equation 3 SLA Strictness definition formula For the normalization intervals for p we have used 99 and 100 as the edges given that these were the ranges encountered in the examined SLAs For q we have used O and 10 minutes as the edges 0 indicates the case where no minimum interval is defined thus abiding by the formula in jError No se encuentra el origen de la referencia and 10 the maximum interval in examined Compute level SLAs However there are larger intervals e g 60 minutes in terms of other layer SLAs Azure Storage The limit to 60 has been tried out in the q case that is included in Table 5 along with the example of the other factors and the overall values of the SLA strictness metrics in the 3 examined public SLAs Project Title ARTIST Contract No FP7 317859 www artist project eu Page 66 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework Table 5 Indicative application of the SLA Strictness metric on existing public Cloud SLAs Provider Ser 99 95 Compute normalized 0 5 normalized normalized 0 0833 0 5 Amazon EC2 1 normalized 1 normalized 99 95 0
126. r s definition from the samples e tcaninstruct users as to whether their current usage application deployment of Cloud services is actually eligible for an SLA and can consult on what is needed for this to be realized Project Title ARTIST Contract No FP7 317859 www artist project eu Page 17 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework e tcan be used to centrally gather statistics from multiple users regarding the behaviour of Cloud services thus enabling accumulated statistics and potentially exploitation scenarios through Business Intelligence analytics 1 4 Fitting into overall ARTIST architecture The generation of the Cloud models is expected to be a semi automatic process In the beginning in order to create a model from the provided meta model one should use the meta model definition and select populate the different required and optional fields In the context of Work Package 7 we have created such models for the most popular Cloud providers like Amazon Azure and GAE Finally the performance aspects of the models can be acquired through the benchmarking suite provided also by Work Package 7 This benchmarking suite will be responsible for installing the necessary benchmarks on the target platforms Tests can be launched and results will be automatically stored in the model that will be available through the ARTIST Repository The overall cloud
127. r file or used in a per case base inside existing code In order for the latter to happen the following piece of code may be used InputStream input null Properties prop2 new Properties try input new FilelnputStream 3alibconfig properties load a properties file prop2 load input abstracted part differences are hidden inside the implementation of each Auditor AvailabilityAuditor thisauditor new AvailabilityAuditor thisauditor setProviderName prop2 getProperty ProviderName thisauditor setServiceName prop2 getProperty ServiceName thisauditor setUser prop2 getProperty user thisauditor setApikey prop2 getProperty APIkey thisauditor setDbhost prop2 getProperty databaselP thisauditor run catch IOException ex I Project Title ARTIST Contract No FP7 317859 www artist project eu Page 71 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework ex printStackTrace finally if input null try input close catch IOException e e printStackTrace For calculating the availability in a given interval the user will be prompted to enter the dates in the following format Figure 37 Enter start year 014 Enter start month 1 Enter start day of month 1 Enter stop year 2014 Enter stop month 2 Enter stop day of month Figure 37 3ALib Availability Calculation Interval Def
128. raph analytics e Data analytics e Software testing e Web search e Web serving Project Title ARTIST Contract No FP7 317859 www artist project eu Page 48 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework We have tested the release 1 0 concerning the first component on Linux the ARTIST repository contains the tool included the prerequisite software packages and the related test results 4 2 4 4 Data Caching Traditionally main storage is too slow to meet the QoS requirements by modern applications in order to solve this kind of problem most of today s server systems dedicate separate caching servers that store informations in their DRAM CloudSuite Data Caching is an open source benchmark used to measure QoS in aforesaid context basically such application uses Memcached data caching server and Libevent a library necessary to its execution to simulate the behaviour of a Twitter caching server using the twitter dataset The metric of interest is throughput expressed as the number of requests served per second A sample output of Data Caching tool output is shown in Figure 24 D te ee ey em EE TI DAT 1 M I 7 s he zd LES BCREE LSZ TEIARQUE OLE iE Beran TEE mire D ran 115i 71 Se TOT EEE Figure 24 CloudSuite Data Caching output example 4 2 4 5 Filebench Filebench 10 is a very flexible file system and storage benchmarking tool Basically it is
129. rf StructuredGnd 3DCurl Dwarf GraphTraversal Quicksort CloudSuite MediaStreaming GetShortHi Figure 52 Illustration of the universal enumeration with the default workloads Project Title ARTIST Contract No FP7317859 www artist project eu Page 115 of 129 D7 2 2 Cloud services modelling and performance Version v1 0 Final Date 15 09 2014 analysis framework In the following table a detailed description of the workloads included in the WorkloadType enumeration is given Beer WE Filebench Fileserver Emulates simple file server I O activity Filebench Varmail Emulates I O activity of a simple mail server that stores each e mail in a separate file var mail server er Filebench Webproxy Emulates I O activity of a simple web proxy server Filebench OLTP A database emulator DaCapo Avrora simulates a number of programs running on a grid of AVR micro controllers DaCapo Batik produces a number of Scalable Vector Graphics SVG images based on the unit tests in Apache Batik DaCapo Jython interprets pybench Python benchmark DaCapo Luindex Uses lucene to indexes a set of documents the works of Shakespeare and the King James Bible DaCapo Xalan transforms XML documents into HTML ones CloudSuite Datacachi a simulation of Twitter type workload for in cache memory data ng Twitter CloudSuite MediaStr It consists of two main components a client and a server the client eaming GetShortHig component emulates real
130. rmance metrics in a homogeneous and independent way without regards to those published by the providers Metrics can then be used to collect over the time quantitative information about the performances offered and constitute the baseline for the selection of Cloud services during the optimization of the whole business process These results will be eventually directly injected into the Artist models representing providers thereby enriching the modelling capabilities of the meta model and covering a significant lack existing until now 1 3 3 3Alib The Abstracted Availability Auditor Library 3ALib innovations can be summarized as follows e Itis an abstracted library meaning that a single method is exposed to the user to utilize for measuring the SLA levels of their services independent from the used provider while hiding in the backend the differences between provider APIs through the use of Apache JClouds library This enables its ease of use e t adapts to each provider s definition of the SLA a very important aspect for someone to claim compensation from the respective provider These definitions have significant differences and preconditions that are checked per provider in the library through the implementation of respective drivers e It logs the availability samples that are needed as forensic evidence by the providers for the acknowledgement of the violation and calculates the availability levels according to the provide
131. rofile In order to define measured availability in CloudML ARTIST the insertion of both fields GenericAvailability and ProviderSpecificAvailability is needed The first one is related to the generic definition followed by Cloudsleuth based on which we can compare different services on actual availability statistics Generally this definition is based on the number of samples acquired which can be summarized as follows TotalSamples UnavailableSamples Availability TotalSamples equation 1 For the population of this metric we will utilize the information provided by Cloudsleuth The second one is related to the provider and service specific definition and measurement of availability as stated in their respective SLAs Moreover for the description of each provider s specific availability the addition of a new field in the availability template is required This new field is ProviderSpecificAvailabilityStatsType and can be attached to each service offering that is bounded by an availability related SLA The overall Availability measurement stereotypes appear in Figure 47 metaclass Stereotype Stereotype Stereotype AvailabilityElement ProviderSpecificAvailabilityStatsType GenericAvailability GenericAvailabilityStats Type ProviderSpecificAvailability ProviderSpecificAvailabilityStats Type 0 1 Stereotype GenericAvailabilityStatsType FailedPercentageOfSLAs Real 1 TimePeriodlnDays Integer 1
132. rry out the migration project into one concrete platform or to be able to propose the target platform that fulfils the migration goals More concretely it is envisioned to require at least the following data from the target platform model 4 Database type In order to check whether or which of the target platforms fulfils the user requirements expressed through the migration goals e g multi instance RDBMS multi tenant RDBMS NoSQL RDBMS This information will be also useful to determine the complexity of migrating from the current database technology to the one required by the user considering the database technology ies supported by the target platform 5 Multi tenancy support In order to check if or which of the target platforms fulfils the user requirements expressed through the migration goals e g One Middleware One instance One Middleware Several instances virtualization by tenant This information will be also useful to determine the complexity of achieving the required multi tenancy level considering the multi tenancy level supported by the target platform 6 SLA requirements amp performance indicators The Technical Feasibility Analysis will select the most appropriate target platform if the decision is not already made by the user or will check if the selected target platform fulfils the user requirements 7 Authorization services in the Maturity Assessment the user will express her needs in terms of authorization mec
133. s Once the design environment has been installed it is necessary to download Cloudml Q Artist project from the github repository in order to be able to start creating models by using the uml profiles defined in the meta model Last version of Cloudml QArtist can be downloaded from next url https github com artist project ARTIST Modeling tree master Cloud 20Provider 20Metamodel CloudMI 40Artist There can be found both the CloudMI Artist eclipse project and also the CloudMI Artist plugin As stated out in section 2 for now we have created profiles for Amazon EC2 Windows Azure and GoogleAppEngine cloud providers In this section we will go through the steps to create a profile for a new cloud provider using Cloudml Artist more concrete an laaS provider in the example For this it is needed to e Import Cloudml Artist previously downloaded from github repository project into Eclipse Eclipse with the aim of complete it with the new profile As can be seen in next screen the meta model is structured as a set of uml profiles stored under two folders main profiles and supporting profiles 4 gt eu artist migration tes cloudml artistvl2 0 3 libraries 4 gt main profiles 3 amazoneczZ profile core profile P googleappengine profile 3 windowsazure profile gt supporting profiles e Create a new Papyrus project using the CloudML O ARTIST meta model as follows Project Title ARTIST Contract No F
134. s ArchitectureTypes Enumeration type on the right Project Title ARTIST Contract No FP7 317859 www artist project eu Page 92 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework laaSStorageService Enumer akion AvailabiityZones Raw_storage East US Volume storage North Central US Object storage Stereotyper South Central US Block blobs aaSStorageService West US Page blab type laaSStorageTypes 1 East Asi capacity GB Real 1 E gie T Tables region AvailabilityZones 1 eA Queues replication type StorageAccountReplicatin Types L North Europe West Europe Enumeration Japan_East storageAccountReplicatin T Japan_West Geo Redundant Read Access Geo Redundant Locally Redundant This stereotype allows modelling storage capabilities offered by an laaS provider As laaSProcessingService it extends from Service stereotype Next are described the properties contained in the stereotype C Relation of the different types of storage offered by the provider Values can be selected by using laaSStorageTypes Enumeration data type on the right part of the picture laaSStorage Types capacity GiB 1 Maximum amount of storage the provider can offer GB 1 List of regions where the data can be stored Values can be selected by AvailabilityZones enumeration data type um replication type StorageAccountRep
135. s offered by cloud platforms The meta model is realized as an extension to the UML meta model Therefore it has been realized in terms of a profile collec tion of profiles regarding specific aspects e g Availability concepts Appendix B Performance Concepts Appendix C etc Profiles created starting from this meta model Appendix F will be used in ARTIST during the migration of an application in order to select the target platform that matches best the requirements and functionalities needed by the re engineered applica tion The second prototype consists of a software suite for benchmarking cloud resources in order to extract performance related data and to include it in the cloud models Since performance aspects are considered as key criteria that are taken into consideration in the selection of the target platform the availability of this data in the cloud models simplifies and makes more accurate the migration phase The software suite includes an installation configuration and execution tool incorporating a set of third party benchmarking tools specifically selected for their effectiveness in evaluating cloud resources performances a database for the data stor age and a user interface to automate the management of executions and results of tests It also includes a GUI for end users to visualize the results of the performance experiments The third prototype consists of an abstracted software library for the measurements of availa
136. s stricter approach Stricter implies that it is more difficult for a provider to maintain this SLA In order to define such a metric initially one needs to identify what are the critical factors that may affect strictness levels These factors are more highlighted in D7 1 based on the SLA analysis from existing public Cloud services Factors should be normalized to a specific interval e g O to 1 and appropriate levels for them may be defined Indicative factors may include Project Title ARTIST Contract No FP7 317859 www artist project eu Page 65 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework e Size of the minimum continuous downtime period Quantum of downtime period q A higher size means that the SLA is more relaxed giving the ability to the provider to hide outages if they are less than the defined interval The effect of such a factor may be of a linear fashion e g 1 q Necessary edges of the original interval before the normali zation process may be defined per case based e g on existing examples of SLAs e Ability to use the running time of the services and not the overall monthly time de noted by a Boolean variable t This would be stricter in the sense that we are not con sidering the time the service is not running as available The effect of such a factor may be of a Boolean fashion 0 false 1 true e Percentage of availability that is guaran
137. sage e Section 8 and 9 describe the Model Updater implementation and usage e Appendixes contain the main profile introduced in the ClooudML ARTIST meta model and an example of how it can be applied to describe Windows Azure 1 3 Main innovations 1 3 1 CloudML meta model CloudML Artist meta model has been created in order to respond to two existing needs at ARTIST project level On one side modelling cloud providers in order to select the most appropriate depending on the specific service needs of an application On the other side modelling the deployment characteristics and values of an application in a concrete cloud provider once it has been selected To accomplish this CloudML Artist is composed of an extensible set of UML profiles through which it is possible to model the main features of cloud providers as well as modelling capabilities to describe services that can be offered under the saas model These same profiles can be used also to specify values related to concrete deployments Project Title ARTIST Contract No FP7 317859 www artist project eu Page 16 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework Despite having many aspects in common with CloudML meta model created in the context of MODAClouds PaaSage and REMICS FP7 projects for multi cloud systems CloudML Artist presents some innovative elements that extend the capacities offered by CLoudML
138. sation claims together with the aforementioned logs In order to enable all this functionality the user only needs to run the provided executable after he has created a local configuration file as described in Section 7 with the necessary fields Alternatively if the functionality needs to be included in a Java program the user must only create one object of the type AvailabilityAuditor and set the properties for it from the configuration details and start the monitoring thread Logs are provided with the detailed measurements of availability and are stored in a backend MongoDB database that can be de ployed anywhere Login credentials for this DB are expected to be provided by the library user upon configuration A separate DB system was selected in contrast to local log files for a vari ety of reasons e Better separation and decoupling of the functional parts of the system e Ability to have a centralized repository of availability information from multiple users e Possibility to run business intelligence queries or in general analytics over the results e Enablement of exploitation scenarios in which a 3rd party service may offer the afore mentioned capabilities and provide insights into users or Cloud providers 6 2 Technical Description 3ALib is a Java library that uses Apache Jclouds API as a means to abstract from the specifici ties of Cloud providers APIs and data format Furthermore it implements specific drivers for each pro
139. scription acquisition AcquisitionType 1 Variety of options on demand spot or reserved pricing discount DiscountType 1 Potential discounts AcquisitionType CostDataTypes Enumeration Units Currencies USD EUR GB TB Stereotype AcquisitionT ype E GA BQ IPAddress BillingMetric Stereotype OnDemandAcquisitionT ype name String 1 type PriceType 1 Stereotype ReservedAcquisitionType name String 1 type ReservedPriceType 1 Stereotype SpotAcquisitionT ype name String 1 type SpotPriceType 1 Stereotype Stereotype Stereotype PriceType ReservedPriceType SpotPriceType price Real 1 price PriceType 1 min Real 1 currency Currencies 1 durationInDays Integer 1 max Real 1 perUnit Units 1 price PriceType 1 Project Title ARTIST Contract No FP7317859 www artist project eu Page 119 of 129 D7 2 2 Cloud services modelling and performance Version v1 0 Final Date 15 09 2014 analysis framework Furthermore for a service acquisition there may be a variety of options for example for Ama zon services there may be on demand spot or reserved pricing For this reasonwe have creat ed the AcquisitionType as shown in the figure above to indicate the various options along with their specifics The final price element is of the basic PriceType which aims to capture cost valu
140. sees Henne n nennen 89 FIGURE 44 JAAS STEREOTYPES I eee eee e nnenne nenne nnsse n nse rese sisse se sese see esee es eren 9 FIGURE 45 PAAS STEREOTYPE G ccccccssccccsscccsssccssscececcecssecuseseueeseecescncuscecusseeeseeescacesescussseueseeesenes 98 FIGURE 46 SOFTWARESERVICE STEREOTWVEPE I eee hene nene nnne e nennen eene enses e nne 102 FIGURE 47 AVAILABILITY MEASUREMENT STEREOTYPES eee eee ee mener nnne nns 105 FIGURE 48 GENERAL INFORMATION OF SLA DEFINITION REGARDING PAAS IAAS AND STORAGE Project Title ARTIST Contract No FP7 317859 www artist project eu Page 8 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework PROD NS 107 FIGURE 49 ILLUSTRATION OF DISCOUNT TYPE AND DIFFERENT ENUMERATION FIELDS THAT COMPLETE THE SLA DEFINITION MODEL se 109 FIGURE 50 ILLUSTRATION OF THE BASIC PART OF PERFORMANCE MODEL AND THE OCL CONSTRAINT EE 112 FIGURE 51 BENCHMARK RESULTS INCLUDED IN PERFORMANCE PROFILE ee 115 FIGURE 52 ILLUSTRATION OF THE UNIVERSAL ENUMERATION WITH THE DEFAULT WORKLOADS 115 FIGURE 53 COST PROFILE STEREOTYPES cccccsscccsesscccsscscsscecscsenecsceesscscnscscacsenecsenevacsenscscussenacses 118 FIGURE 54 WINDOWS AZURE MODEL MAIN STEREOTYPES eseeeee eee ee n Henne nnne nnns 125 FIGURE 55 WINDOWS AZURE INSTIANCETVPES He he e nennen ressent nsns nere eese serene 128 Project Title ARTIST Contract No FP7 317859
141. sible for collecting and parsing results and storing them in the ARTIST repository The GUI will consist of four main areas e The test selection tab area This will be used by the user in order to select one of the benchmarks to run It will also lead to different configuration pages in the second area Project Title ARTIST Contract No FP7 317859 www artist project eu Page 39 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework e The test configuration area In this part the user will insert the needed configuration parameters for each test e g the number of iterations periodicity of test DB size etc e The target selection area This will be used to insert details for the target services e g provider details login account details etc e The results area This will be used to communicate with the repository and store the results and can also be used to perform queries on the measurements The GUI form for filtering historical data is depicted in Figure 9 The GUI form for retrieval of information is depicted in Figure 10 The GUI is provided in its final integrated form in M30 of ARTIST SUT Lag End Dare Date cL Figure 9 Benchmarking GUI Filtering of Historical Data Project Title ARTIST Contract No FP7 317859 www artist project eu Page 40 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analy
142. sis framework The operation completed successfully Reports 2388275 00 103838 04 11 Chart data Test Started Succeded Total Failed Done Errored Date Number Requests Request Requests Requests Requests Requests Test1 100000 97554 100000 2446 100000 0 2014 03 28 Test2 100000 89018 100000 10982 100000 0 2014 03 28 Test3 100000 97196 100000 7804 100000 0 2014 03 28 Figure 10 Retrieval of information and depiction 4 2 2 3 1 GUI Controller Integration As far as the integration between GUI and controller concerns it has been obtained by linking the GUI Reporting form with the Artist benchmarking database containing whole the experimentations results In particular the user can select the test period of interest the cloud test as is the benchmark application field e g Java Applications and the Instance types In the following the target instance types that have been object of benchmark grouped by Provider Cloud Provider mismall X xx pp L1 1l II J pmimedum xX d X TT E dE EM XE gib X ho o J gt d O re 0 m dq 1H Jj mixage x xp InstanceTypes pp pp o picpicpu PP X TI EE GE SR EE 23Gh2CPU PP XJ p de d eee eee 4Gb2CPU X jJ Lo 170 mJ O HN Jj 4Gb3CPU TK eme EEG 4Gb4CPU P X J LL 1 BR I C Jo J dealt d X J 4 X gt Figure 11 InstanceTypes for Cloud Provider Project Title ARTIST Contract No FP7 317859 www artist project eu Page 4
143. t Software Packet Manager ARTIST Tools Eclipse Figure 8 scenario for the ETICS and ARTIST repositories An example of download link for YCSB 0 1 4 tool for CentOS platform from the ETICS Repository is the following http etics res eng it eticssoft repository artist artist ycsb 0 1 4 centos6 x86 64 gcc447 artist ycsb 0 1 4 1 centos6 x86 64 rpm 4 2 2 2 Benchmarking Controller The Benchmarking Controller component is in charge of managing executions of benchmarks The main objective of this component is to relieve the user from the usual work flow of benchmarking execution that needs to be done manually 1 creation of target environment 2 installation of benchmarking tools 3 execution of benchmarks and 4 retrieval of results 5 data storage Complete automation of the work flow will be possible only on certain combination of cloud provider and benchmarking tool In all the other cases some step will still require manual intervention of the user For instance if an laaS cloud provider does not provide an API to create destroy virtual machines In this case the user has to create the environment manually and then provide the endpoint of the virtual machines to the Benchmarking Controller that will continue the work flow 4 2 2 3 Benchmarking GUI The benchmarking GUI is the web front end for launching benchmark tests against target ser vices by using the predefined benchmarks per application category It will also be respon
144. t 1 Percentage of discount discountMetricRelevant Currencies 1 Currency on which it is ToAmount applied the discount periodOfMinMaxCalcul Integer 1 Period of time while is valid ationInDays the discount Project Title ARTIST Contract No FP7317859 www artist project eu Page 122 of 129 D7 2 2 Cloud services modelling and performance Version v1 0 Final Date 15 09 2014 analysis framework Appendix E Security profile Another crucial aspect of Cloud environments is their ability to scale for meeting increased demand However this scaling does not come cheap and in the event of a Denial of Service attack the resulting costs could be significant for a service offering Thus the ability of a Cloud provider to successfully meet this type of attack on e g a network level with the usage of suitable filtering mechanisms etc is critical But even if the issue of raising costs is not applicable the performance of the offering itself and indirectly the performance of the provider countermeasures should be tested when it consists a target for a DoS attack Next a picture containing stereotypes and data types related to this profile DoS resillience Enumeration doS Type DoS Type 1 DoS Type timeToLiveUntilCrash MeasurementType 1 4 pingingDelay MeasurementType 1 4 sizeOfAttack Integer 1 Name Type Card Description dos Type Dos TypeType 1 DoS attacks ICMP SYN Teardrop timeTo
145. t Title ARTIST Contract No FP7317859 www artist project eu Page 113 of 129 D7 2 2 Cloud services modelling and performance Version v1 0 Final Date 15 09 2014 analysis framework DwarfsResult stereotype includes the problem size which is set as a real parameter and the execution time for test completion In the following table the aforementioned parameters are summarized ame e emo rom score 0 1 Runtime benchmark result 0 1 Problem size size FilebenchResult stereotype has been defined in order to capture the typology of results includ ing the various statistics that are returned In the following table the aforementioned parame ters are summarized ame e mm Dem mm ops Del The number of opera tions ond rw 0 1 Reads writes to get a feeling for maximum performance cpuOp 1 Number of cpu oper ations NM co EE gt Regarding the Cloudsuite case which offers a benchmark suite for emerging scale out applica tions eight application types we have kept only the generic average score to be included in the model instances in order to simplify the descriptions each of the applications reports a large number of statistics that are case specific Next the CloudSuiteResult stereotype is described ame e Yee remm Average score DI Average score related to specific Project Title ARTIST Contract No FP7317859 www artist project eu Page 114 of 129 D7 2 2 Cloud services modelling and perform
146. teed Again this may be of a linear fashion with logical intervals defined by the examined SLAs e Existence of performance metrics e g response timing constraints This may be a boolean feature x however its values may be set to higher levels O or 5 The im portance of this will be explained briefly The added value of such a metric may be in the case we have to deploy applications with potentially different characteristics and requirements as one would expect For example having soft real time applications would imply that we definitely need to have feature 4 Other less demanding applications may be accommodated by services whose SLAs are less strict Thus suitable value intervals may be adjusted for each feature If we use a value of 5 for the true case of feature 4 and all the other features are linked in such a manner that their accumulative score is not higher than 5 then by indicating a necessary strictness level of 5 implies on a numerical level that feature 4 needs definitely to be existent in the selected Cloud service Depending on the application types and their requirements and based on the metric definition one can define categories of strictness based on the metric values that correspond to according levels e g medium strictness needs a score from 2 to 3 etc It is evident that such a metric is based only on the SLA analysis and is static if the SLA definition is not changed Thus they can be easily included in the pr
147. tfilel 128311ops 213B8ops s 0 0mb s 0 0ms op 2320us op cpu Oms Oms deletefilel 128316ops 2138ops s 0 0mb s 0 2ms op 2535us op cpu Tome 458m3 closefile3 12783230ps 21390ps s 0 0mb s 0 0ms op 2328us op cpu Oms Oms readfilel 12832 7ops 21390ps s 283 8mb s 0 1ms op 2460us op cpu Tome 267ms openfile 12832 50ps 21398o0ps s 0 0mb s 0 0ms op 2332us op cpu Oms 2ms closefile 1283320p3 21390p83 3 0 0mb s 0 0ms ap 2332us op cpu O0ms Oms appendfilerandl 128337ops 21390ps s 16 6mb s 0 1ms op 2377us op cpu Tome 559ms openfilel 1283430p3 2139o0ps s 0 0mb s 0 0ms op 2353us op cpu O0ms 2ms closefilel 1283490ps 213980ps s 0 0mb s 0 0ms op 2317us op cpu Oms ims wrtfilel 1283520ps 21390ps s 265 2mb 3 0 1ms op 2601us op cpu Oms 268ms createfilel 1783580ps 21390ps s 0 0mb s 0 1ms oap 2396us op cpu ms 267ms 12462 67 145 IO Summary 1411677 ops 23526 ops s 2139 4278 r w 565mb s 3983us cpu op 0 2ms latency 12462 67 145 Shutting down processes root user Figure 25 Filebench output example 4 2 4 6 DaCapo In order to evaluate Java based applications can be used the DaCapo benchmark suite it is designed to facilitate performance analysis of Java Virtual Machines compilers and memory management This benchmark suite is intended as a tool for Java benchmarking by the programming language memory management and computer architecture communities It consists of a set of open source real world applications with non trivial mem
148. th_Central_US West_US East Asia Southeast Asia Brazil South North Europe West Europe Japan East Japan West Stereotype ScalabilityMetrics Type bootTime MeasurementType 1 scaleUpTime MeasurementType 1 scaleDownTime MeasurementType 1 Figure 43 CloudMI Artist Core Data Types Contract No FP7 317859 Page 89 of 129 www artist project eu D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework laaS This section describes the laaS uml profile contained in CloudMI Artist meta model This profile is composed of infrastructure related stereotypes and data types and allows in conjunction with higher level stereotypes and data types already described to model main characteristics of any laaS provider laaS providers offering consists typically of a set of services classified in three categories processing services network services and storage services As can be seen in next picture these services are represented by laaSProcessingService laaSNetworkService and laaSStorageService stereotypes Following the picture can be found a complete description of the main entities contained in the profile Project Title ARTIST Contract No FP7 317859 www artist project eu Page 90 of 129 D7 2 3 Cloud services modelling and performance analysis framework Profle a85 stereotypes aaSProcessingservice numProcessors Integer 1
149. the cost of Spot Price For instance Amazon EC2 charges the Spot Price which fluctuates periodically depending on the supply of and demand for Spot Instance Capacity Name Type Card Description min Float 1 Min value of spot price max Float 1 Maximum value of spot price price PriceType 1 Cost of spot price DiscountType Stereotype DiscountType durationInDaysOfPrepurchase Integer 1 prepaid Boolean 1 minApplicableAmount Real 1 maxApplicableAmount Real 1 discountPercentage Real 1 discountMetricRelevantToAmount Currencies 1 periodOfMinMaxCalculationInDays Integer 1 USD EUR GB TB CostDataT ypes Enumeration Enumeration Units Currencies IPAddress BilingMetric This type is defined in order to specify various discounts that different providers offer to customers in relation for example to usage size Name Type Card Description durationInDaysOfPrepu Integer 1 Duration of discount rchase prepaid Boolean 1 Get discount when prepay minApplicableAmount Float 1 Minimum discount limit Project Title ARTIST Page 121 of 129 Contract No FP7317859 www artist project eu D7 2 2 Cloud services modelling and performance Version v1 0 Final Date 15 09 2014 analysis framework maxApplicableAmount Float 1 Maximum discount limit discountPercentage Floa
150. the local database processed and the appropriate graphs are depicted In order to draw conclusions from the execution of the benchmarks one should compare between same colour bars indicating similar workloads From the graphs it is evident that the performance for a specific workload varies and depends on both the type of workload and 4 https code google com p fincos issues detail id 3 5 www modaclouds eu Project Title ARTIST Contract No FP7 317859 www artist project eu Page 53 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework the VM instance size For instance for DaCapo benchmark the workloads performance across Azure A2 Standard Amazon m1 large and Amazon m1 medium is almost similar apart from some cases where Amazon provides better results for workload h2 while Azure for workload avrora Extended information for the benchmarking results is provided at the Deliverable 7 4 Bi avrora Bi eclipse E fop E h2 E jython E pmd E tomcat xalan Figure 28 Performance time in ms for DaCapo workloads Having the performance information extracted from the benchmarking process we create the respective placeholders in the CloudML ARTIST model framework and populate them with the benchmarking results For example in Amazon EC2 case in order to include the performance placeholders in the model the following steps should be undertaken e As alr
151. to the application s needs It extends from Service stereotype included in the core package Next are described the properties contained in the stereotype p RE InstanceTypes Type of instances a PaaS provider can offer Possible values given through InstanceTypes enumeration entity whose values can be seen in the previous picture Name of the instance used for identification purposes memoryOffered i 1 Memory offered by the instance in MB cpuOffered i 1 CPU offered by the instance in GHZ SaaS By this extension implemented through a UML profile as the other extensions of CloudML it is intended to provide the possibility to model software as a service SaaS features thus covering along with laaS and PaaS profiles the needs of the three flavours of cloud computing Taking into account its flexibility to be extended if new needs arise the current version of the profile allows to model different software services by specifying authentication methods to access the services invocation urls interfaces through which the service can be invoked and also configuration aspects Next a detailed description of the most relevant stereotypes included in the SaaS profile Project Title ARTIST Contract No FP7317859 www artist project eu Page 101 of 129 D7 2 2 Cloud services modelling and performance Version v1 0 Final Date 15 09 2014 analysis framework SoftwareService In order to integrate 3rd party serv
152. tualCores 16 memory 112 0 localDisk 127 www artist project eu D7 2 2 Cloud services modelling and performance Version v1 0 Final Date 15 09 2014 analysis framework In the previous image can be seen the set of stereotypes describing the types of instances offered by Windows Azure provider The laaSInstanceType stereotype defined in the laaS profile see Appendix A is applied to all of them The properties contained in the InstanceType permit to specify information such as physical resources offered virtual cores memory local disks supported operating systems and regions where can be run This information is very useful in order to be able to automatically check if a concrete instance could cover deployment needs of a specific application Project Title ARTIST Contract No FP7317859 www artist project eu Page 129 of 129
153. ture It automates execution of benchmarking tests Benchmarking GUI this components has two main functionalities a provides a user interface for submitting execution of new benchmark tests and b provide a user interface for browsing performance data collected Backend Raw Data DB for storing the raw data from the executions A separate component is used in order not to include all the measurement data in the provider models for better manipulation of the latter Thus average values may be included in the models while more detailed queries may be addressed towards the backend raw data The entire architecture constitutes a framework for managing the execution and storage of data of benchmarking tools The modular and flexible architecture allows to incrementally add new tools and automation scripts in the repository extending the coverage of benchmark tests offered by the ARTIST Benchmarking Suite 4 2 2 Components Description 4 2 2 1 Benchmarking Repository The Benchmarking repository component is part of the ARTIST Repository component The advantage of using the ARTIST Repository also for benchmarking is twofold a it is consistent Project Title ARTIST Contract No FP7 317859 www artist project eu Page 37 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework with other ARTIST tools and b makes the content indexable searchable and usable by the other tools
154. ud model generation process 1 4 1 Management of Cloud models and benchmarking results During an ARTIST migration project a great number of different artifacts such as provider descriptions benchmarking tools or benchmarking results are used and produced The ARTIST repository provides a central place to keep and organize these artifacts thus keeping the different work products manageable In addition to storing the artifacts content the ARTIST repository can enrich them with meta data that can be used to further describe them and to organize and find artefacts more efficiently A benchmarking result set can for example be stored together with information about the used benchmarking tool the platform model and used parameters In the ARTIST repository artefacts are organized in collections and packages on a physical level Projects can store their artefacts in their own dedicated collection and configure the access rights to fit their privacy needs There is one special collection for the publicly available artefacts that are featured on the ARTIST Marketplace Inside a collection the artefacts are organized in packages Artefacts can be addressed by their physical location using a URI of the form aar collectionName packageName ArtefactName Benchmark results for the ATOS use case could for example be stored using the following pattern aar UseCaseATOS benchmarks providerName toolName result timestamp Artefacts can also be
155. uisitionType SpotAcquisitionType Je name String 1 name String 1 name String 1 p P T s discount Discount ype HJ type PriceType 1 type ReservedPriceType 1 type SpotPriceType 1 Stereotype Stereotype Stereotype PriceType ReservedPriceType SpotPriceType price Real 1 price PriceType 1 min Real 1 currency Currencies 1 durationInDays Integer 1 max Real 1 perUnit Units 1 price PriceType 1 Discount Type durationInDaysOfPrepurchase Integer 1 prepaid Boolean 1 minApplicableAmount Real 1 maxApplicableAmount Real 1 discountPercentage Real 1 discountMetricRelevantToAmount Currencies 1 periodOFMinMaxCalculationInDays Integer 1 Stereotype AcquisitionT ype CostDataTypes Currencies GB USD Operations EUR Hour GB Day TB Month IPAddress BillingMetric Figure 53 Cost Profile Stereotypes Project Title ARTIST Contract No FP7317859 www artist project eu Page 118 of 129 D7 2 2 Cloud services modelling and performance Version v1 0 Final Date 15 09 2014 analysis framework ServiceCost Stereotypes Service Stereotype 5ServiceCast acquisition Acquisition Type 1 discount DiscountType 1 Parameters for this type include the acquisition type and potential discounts that may be appli cable per case Name Type Card De
156. ureTypes Property through which it is specified if the instance supports 64 bit architecture 32 bit architecture or both Uses ArchitectureTypes Enumeration type on the right networkPerformance NetworkPerformanceTypes This property specify the type of network performance offered by the instance at high level Possible values are Project Title ARTIST Contract No FP7 317859 www artist project eu Page 95 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework detailed in NetworkPerformanceTypes enumeration type autoScaling Boolean 1 Boolean value to specify if the provider offers autoscaling capabilities or not laasFeatures Stereotypes IaasFeatures virtual private cloud offering Boolean 1 Hypervisors xen KVM VMware Hyper V hypervisor Hypervisors 1 This stereotype extends from generic CommonFeatures stereotype and allows specifying specific laaS features Next are described the properties contained in the stereotype ERE virtual private cloud off Boolean 1 Property to specify if the ering laaS provider can offer or not a virtual private cloud VPC hypervisor Hypervisors 1 Property used to specify the hypervisor used by the laaS An hypervisor is a piece of software firmware or hardware that creates and runs virtual machines The possible values can be obtained from Hypervisors enu
157. urity related characteristics at provider level At the moment the amount of modelled characteristics can be significantly enriched Benchmark profile This profile can be included when modelling a cloud provider to specify results of benchmark tests when attached to specific service instance types 2 2 3 Technical Specification The meta model is delivered in the format of a set of UML profiles created with Papyrus plug in for Eclipse IDE It is fully compliant with XMI standard and it is ready to be imported as a profile in any model created with Papyrus Project Title ARTIST Contract No FP7 317859 www artist project eu Page 28 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework 3 CloudML ARTIST Delivery and Usage 3 1 Package information The CloudML ARTIST meta model is available as a set of UML profiles compatibles with the Eclipse IDE and ready to be imported and used to create new models In order to improve the portability the last version of CloudMIQArtist has been encapsulated as an eclipse plugin so the profiles are automatically registered in eclipse 3 2 Installation instructions CloudML ARTIST has been created by making use of Eclipse ecosystem more in concrete by using Papyrus 0 10 1 http www eclipse org papyrus design tool plugin installed inside Eclipse Modelling Kepler SR1 that can be downloaded freely from http www eclipse org download
158. uth Central US Ubuntu NetworkPerformance Var processorArchitecture Architecture Types 1 iiie FreeBSD Very Low networkPerformance NetworkPerformanceTypes 1 ee Asia eben PAN Low autoScaling Boolean 1 Brazil South p Moderate North Europe s High West Europe Japan_East Japan West This stereotype allows modelling the characteristics of the instance types an laaS provider can offer Next are described the properties contained in the stereotype Project Title ARTIST Contract No FP7 317859 www artist project eu Page 94 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework E a a instanceCategory InstanceCategories Category of the instance depending on the features offered Possible values can be selected by using InstanceCategories enumeration type that can be found on the right part of the picture above region AvailabilityZones List of regions where the instance can be run Values can be selected by AvailabilityZones enumeration data type operatingSystem Relation of Operating Systems supported by the instance Represented by the Enumeration type OSs on the right side of the picture Number of cores offered om zm by the instance memory 1 Maximum amount of RAM the instance can offer GB Capacity of the disk at instance level in GB 1 processorArchitecture Architect
159. vider in order to enforce the specific preconditions that are mentioned in their re spective SLAs and the specific calculation of availability followed by each provider Project Title ARTIST Contract No FP7 317859 www artist project eu Page 59 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework 6 2 1 Prototype Architecture The design of 3ALib appears in Figure 32 Following the analysis of the providers SLAs as it appears in D7 1 12 the common concepts and terms are abstracted to generic methods that are used in all providers Specificities are included as separate methods for the enhancement of the library s generic and extensible nature 1 abstracted method to calculate availability AWSEC2 B BESuu Abstracted Mi l based on Concepts Methods provider CETERA BET definition Compute SLAs Provider r m mid rovider Microsoft 8 specific St Azure i Terms Apache JCIlouds AWS API a Azure API Figure 32 3ALib General Design 6 2 2 Components Description The class diagram of the library appears in Figure 33 The Main class may be used only in case the user needs to launch the executable directly or as an example of how to instantiate an AvailabilityAuditor and configure it The Availability Auditor uses one of the separate provider auditors based on the user s selection These individual auditors implement the same interface Auditi
160. viders availability definitions as described in D7 1 it is not feasible to directly compare provider defined availability metrics since these differ in definition For this reason it is more meaningful to either follow a more generic direct definition as done in CloudSleuth for example Equation 1 or abstract to a more generic concept which is the SLA adherence level Equation 2 This can be defined as the ratio of violated SLAs over the overall examined SLAs Since SLA period is set to monthly cycles this may be the minimum granularity of observation TotalSamples UnavailableSamples Availability PI Equation 1 Generic Availability Definition used in CloudSleuth violatedSLAs SLA Adherence B overallObservedSLAs Equation 2 SLA Adherence Metric Special attention must be given for cases that sampling is not continuous indicating that the client did not have running services for a given period applicable for an SLA These cases must be removed from such a ratio especially for the cases that no violations are examined in the limited sampling period given that no actual testing has been performed If on the other hand even for a limited period a violation is observed then this may be included 6 2 5 2 SLA Strictness metric Besides SLA adherence other metrics may be defined in order to depict the strictness of an SLA As a prerequisite we assume that the metric must follow a higher i
161. were linked with spe cific benchmarks that provide application level workloads characterization and raise the ab straction for the end user More information regarding the analysis of these applications and the description of benchmarks can be found in the content of D7 2 and D7 1 respectively In order to incorporate the information regarding performance aspects in CloudML Q ARTIST profile we have created a sub profile that can be attached to the generic description of core profile that is used to describe a virtual offering This subprofile includes a number of different benchmarks covering the most prominent application types and providing the ability for ac quiring performance score results Regarding performance profile analysis the basic stereotype is BenchmarkedElement which includes BenchmarkResult Figure 50 stereotype as an attribute The latter includes only one property and is related to the potential benchmark workloads Moreover in the definition of our performance model an OCL constraint has been generated According to this constraint a BenchmarkElement can be only applied to an InstanceType element In the same way different constraints may be defined throughout the profiles to link them with the core profiles 7 Stereotype RequireslnstanceTypeBaseElement OCL self base Stereotype superClass gt collect e e name gt includes Instancefype l Stereotype BenchmarkedElement result Stereotype
162. world clients sending requests to stress a h streaming server Dwarf StructuredGri Regular grids can be automatically refined d 3DCurl Dwarf GraphTraversa Decision Tree searching quicksort Quicksort Filebench Webserver Emulates simple web server I O activity Project Title ARTIST Contract No FP7317859 www artist project eu Page 116 of 129 D7 2 2 Cloud services modelling and performance Version v1 0 Final Date 15 09 2014 analysis framework Appendix D Cost profile This section describes the Cost profile included in CloudMI Artist that it is used mainly to specify information related to prices and discounts that can be applied to the different services offered by the cloud providers A variety of pricing schemes can exist in modern Cloud infrastructures based on different of fering characteristics means of payment or acquisition methods ServiceCost is a generic type which includes two significant subtypes Acquisition and Discount Next a picture showing the stereotypes and data types included in this profile and a detailed description of each one Project Title ARTIST Contract No FP7317859 www artist project eu Page 117 of 129 D7 2 2 Cloud services modelling and performance Version v1 0 Final Date 15 09 2014 analysis framework Stereotype Service ZN Stereotype Stereotype Stereotype Stereotype ServiceCost OnDemandAcquisitionType ReservedAcq
163. ying name for the provider ProcessingOffering This stereotype is used to categorize those services representing the processing capacities offered by the provider such as number and speed of processors processors architecture and SO On StorageOffering This stereotype is used to categorize those services related with the storage capacities offered by the provider As can be seen on sections related to laaS and PaaS profiles below in this appendix these storage services can be described by different sets of properties depending mainly on whether they are infrastructure or platform related services NetworkOffering A network offering is a named set of network services a cloud provider can offer such as e DHCP Project Title ARTIST Contract No FP7 317859 www artist project eu Page 83 of 129 D7 2 3 Cloud services modelling and performance Version v0 6 Final Date 31 03 2015 analysis framework e DNS e Source NAT e Static NAT e Port Forwarding e Load Balancing e Firewall e VPN PlatformOffering This stereotype is used to categorize those services specifically offered by a paas provider such as preinstalled application servers application frameworks programming languages and so on SaaSOffering This stereotype is used to categorize those services offered by a saas provider Next on the section related to the SaaS profile can be found a description of the services SoftwareService entity a SaaS prov
164. ype of access to the data storage Possible values given through DatastoreAccessType enumeration entity whose values can be seen in the previous picture PaaSSoftwareOffering Project Title ARTIST Contract No FP7317859 www artist project eu Page 99 of 129 D7 2 2 Cloud services modelling and performance Version v1 0 Final Date 15 09 2014 analysis framework Stereotype PaaSSoftwareOfferin applicationC ntainerService stereotype sStereotypes RuntimeService ApplicationContainerService name Runtimes 1 name ApplicationContainers 1 programmingLanguaje ProgrammingLanguages 1 applicatianFramework ApplicationFrameworks 1 This stereotype allows modelling the software offering related to a PaaS provider It extends from PlatformOffering included in the core package The PaaSSoftwareOffering consists of a set of services that can be offered by a PaaS provider with the aim of making easier the deployment of applications on the servers Through his offering the providers offer preinstalled software environments such as frameworks and application servers so the developer doesn t have to install them in the production environment he she can select those services fitting the needs of the application and use them directly As can be seen in the picture for now we have included two types of software services that can be offered under this categoriz
165. ysis framework 10 Conclusions This document is delivered along with the final release of four prototypes in Work Package 7 a the CloudML ARTIST meta model and b the Benchmarking Suite c the 3Alib library d the Model updater CloudML ARTIST is an extension of the CloudML meta model with several new stereotypes and relationships to describe aspects of cloud providers offering not taken into consideration by CloudML In particular we added stereotypes to capture a laas PaaS and SaaS level aspects b performance aspect and c pricing security and availability aspects CloudML ARTIST is available as an UML meta model ready to be imported in Eclipse Papyrus plug in to be used as profile for cloud models The concepts of the meta model and of the supporting profiles have been used in practice to model provider features and capabilities This process was continued in Y3 in order to identify improvement aspects enrichment of information and extension of the produced models In this work input from WP9 from the usage of WP7 models has acted as a valuable feedback for features that should be reformatted extended or removed Furthermore specific addons have been incorporated including aspects of provider benchmarking and inclusion of the relevant information in the models Of particular interests are performance aspects since they will be a key criteria both in the target environment selection and migration evaluation processes For this reason we

D7.2.3 Cloud services modeling and performance

Contents

Download Pdf Manuals

Related Search

Related Contents