Home

D3.5.3 - MODAClouds

1. URLs Project s modules Modules installed in the project Expand modules catalog to add modules to the project Enable Scope Name Version Status Y User O Modeler Module 8 1 10 9015 Started Modelio version Compatibility aDe Compatible dh ModelerModule 8 1 10 Compatible dh WebModelPublisher 4 0 01 Compatible a Show only latest versions Y Show only compatible versions Public Final Version 1 0 Dated 31 03 2015 27 MODAClouds MOdel Driven Approach for design and execution of applications on multiple clouds Deliverable D3 5 3 Open the Module Configuration page in Configuration menu Click on to expand the Modules catalog In the Modules catalog select the module you want to install Click on to install the module in project A WN Adding the Resource Model to a Project 1 Open the Configuration Libraries menu option 2 Click on the Add button on the Local libraries tab 3 Select the ResourceModel _1 0 0 ramc file 4 Click on the Add to project button Add local library x Add a local library to the project Install a new model component File path Users malmeida Dropbox Projects MODAClouds WP9 ModelRepository t cs Model component caracteristics Name ResourceModel Version 1 0 00 Version history Dependencies Contributing modules ModelerModule 8 2 04 Imported files D Cancel Add to project Model an application a
2. 3 M36INTEGRATED PLATFORM TEST BEDS cecsissssiccvissisiscsasessstinonsscersstssastedscdudenieasva0vcnsrseetecterserszerueddseorsisssedes 18 3 1 PLEXIANTES Oir O Nate dde 18 3 2 IFAT TESTBED teirinn n E AA O A 20 4 MODACLOUDS RUNTIME PLATFORM BUILD INSTRUCTIONS nssssusnunnunnunnunnunnnnnnnnnunnunnunnnnnnnnnnnnnnnnnnnnnnnan 21 4 1 BUILDING RPM Scans ei aa 21 4 2 IVI AIAG DERCOYMENT osas drid 22 A OO 22 A A O O 23 kunning the COMPONEN tradi id dia bend 23 Updating rom VOC OV ZO diia io 23 4 3 AUTOMATIC DEPLOY MEN DINO SAUL Cira Ai 24 5 CONCLUSION erie anana cca ow cases cc ea assess gnc aasa anc obs aaea na casas aoaaa AU 24 6 REFERENCES soiien AAA A A A AA A A A AAA 24 A MODACLOUDS INSTALLATION AND USAGE cccccsscsssesseesseesenesseesecnneessneseneesnessnesnneenenesensssnessaesnneessnenenessness 25 A 1 CREATOR 4CLOUDS INSTALLATION AND USAGE GUIDE sin dad dial 25 ero oe RE Uo a ey q POPE PAE ERE E ven Te mrs RU RTT OnE A 25 TCV FUNICEIONS OF Creator A Clouds sialic ea A aa ne o cedo tea et lo es 25 Creator CHOU S VCC o ains 26 lostalo Creator Fh CLO UO reacios AAA a aca 26 Adding modules from the Modelio modules catalogue cicicicinioninininmr 26 Creating U New PEO CA oie 26 PUG TAG GO module toa Droe Cua r a dank ie sushi leases dead divas ataiid 27 Adding tne Resource Modeltosd ROJO Caria a ERAN ica 28 Modelanapphcation at CCIM VAL A AS 28 Modelanapphcationat CCPM level did 29 Mode DOS Constraints AAA iia 31 GenerateinonitorINONBULOS cist
3. 4 1 Building RPMs While the full detail of how MODAClouds components services can be built from RPMs is presented in D3 4 2 and covers management of MODAClouds RPM packages building the package JSON descriptor and details on wrapper scripts it is proficient to summarise the main aspects to accompany and contextualise this final integrated solution The developer of each MODAClouds component needs to provide the following artefacts e a distribution bundle namely distribution tar gz archive that contains the files needed to run the component e adescriptor in JSON format namely package json used build the RPM i e packing the distribution and the wrapper script e a wrapper script namely service run bash that runs the service The first step of packing procedure is building the distribution bundle This is done by the developer and the tools used to do this largely depends on the environment Java Python etc in which the application has been developed This artefact together with the wrapper script are next packaged into a RPM package with the help of Public Final Version 1 0 Dated 31 03 2015 21 MODAClouds MOdel Driven Approach for design and execution of applications on multiple clouds Deliverable D3 5 3 the mos package builder py script mOS package builder tool and the package json configuration file which is specific to each component Once the RPM is built it is deposited under http mos repositories mosaic apps e
4. Create QoS requirement 2 Move the created constraint to the instance level deployment diagram In order to make it refer to a model element create a dependency link between it and the element Then add a lt lt targets gt gt stereotype to the dependency see below Public Final Version 1 0 Dated 31 03 2015 31 MODAClouds MOdel Driven Approach for design and execution of applications on multiple clouds Deliverable D3 5 3 QoS Constraint CloudNodelnstance SampleCloudNode 3 On Demand Micro Instance Virtual Hardwares R Costs Cost Profiles QoS constraint added to the instance deployment model 3 In order to edit the parameters of the constraint click on each of its attributes and then use the element view to set its Value see below E Outline FU Audit Diagra 2 Link Ed B Script ES Elemen 2 Notes a Symbol E Persista a UML E 8 JavaDesigner PI rangelnSet Modeler Module Type string from UML Types i Visibility Public I Multiplicity min 1 Multiplicity max 1 Value Access mode Read Write Modelling QoS constraints Generate monitoring rules 1 Generate monitoring rules from the QoS constraints To do so right click on the deployment model then select Creator 4Clouds gt Model transformations gt Generate monitoring rules from QoS constraints The generated rules should appear under Monitoring rules under CPSM models 2 You can als
5. Space 4Clouds component The latter can be used to estimate the performance of the identified solution e g response time and to find a cost optimised multi cloud deployment configuration At this stage an iterative process may start to tune the application design until a suitable solution is identified The output of this process is a CloudML deployment model that can be used by the application provider to automatically deploy the multi cloud application Exploitable Module Component Integrated Asset e Ts Functional Modelling Tool Modelling Tool Integrated Space 4Clouds Integrated MODAClouds IDE LINE Integrated Creator 4Clouds CloudML 4Clouds Resource Repository Integrated Resource Repository Table 3 Creator 4Clouds Components As said before at design time Creator 4Clouds provides first support for interactions with Venues 4Clouds This interaction consists in exchanging a resource extension container and a Palladio resource environment describing the high level assets that compose the cloud application Creator 4Clouds obtains in return an updated resource container extension that contains the types of providers services and resources to be used by the application It can then interact with Space 4Clouds to analyse application models It sends a set of 10 types of Palladio models and extensions to Space 4Clouds and obtains in turn an XML file describing the type of providers and resources to be used at the CPS
6. 2015 43
7. 4 On the instance level model use the explore view and the diagram view to define the actual deployment architecture Define in which cloud resources components will be deployed You can also use the element view to define low level information on your instances see below E Outline Fu Audit s Diagra 24 Link Edi B Script Ee Elemen 23 gt m Notes a Symbol El Persista a amp KT A Sun O Modeler Module minRAM El Node maxRAM minCPU maxCPU minStorage OS location sshKey securityGroup groupName privateKey Element view at CPIM instance level 5 When you are done generate CPSM level model To do so right click on the deployment alternative then select Creator4 Clouds gt Model transformations gt Generate CPSM deployment alternative 6 First of all for each cloud provider you need to use define a cloud account for it To do so right click on the Cloud account element under the CPSM deployment model then select Creator4 Clouds gt Elements gt Create provider account 7 You can also use the element view to define the properties of each provider account illustrated below In order to fill these fields consult CloudML documentation E Outline FU Audit gt Diagra 2 Link Edi B Script 3 Elemen amp 3 gt m Notes a EX Symbol E Persista m lr Rod Y ES UML Modeler Module CloudAccount provider name provider credential provider endpoint provider properties Elemen
8. Builder Code https github com ieat mosaic mos package builder Deliverable D6 5 3 Documentation https wiki volution ro Mosaic Projects MosPackageBuilder Guide License Apache 2 0 mOS Package Builder is the component that offers a simple way of building mOS packages It offers a set of scripts and content description that is needed in order to prepare MODAClouds developed tools to be deployed automatically on OpenSUSE Tower 4Clouds Tower 4Clouds is the integrated monitoring platform of MODAClouds responsible for collecting analysing visualizing and storing monitoring information at runtime It relies on monitoring rules generated from Creator 4Cloudsdefined at design time and collects metrics from the monitoring components as well as generates statistical inference on the collected data specified in the monitoring rules Data Collectors deployment and configuration is described using CloudML and deployed using the Models Runtime engine together with the application They are informed about the Manager endpoint by the Models Runtime engine Data Collectors can then periodically contact the Manager to synchronize their configuration The Manager is in charge of offering APIs and a GUI for installing Monitoring Rules and attaching observers to metrics The Manager instructs Data Collectors on whether how and where Data Analyzer endpoint to send monitoring data The Data Analyzer is in charge of aggregating verifying conditions and gene
9. Collectors Mathlab Weka SDA o Space 4Clouds consisting of Self Adaptation Reasoner Self Adaptation Stress Tester Load Balancer Reasoner Models Runtime engine o ADDapters 4Clouds consisting of Data Migration and Synchronization Load Balancer Controller Object Store Artefact Repository Batch Engine mOS Image Builder mOS Package Builder 1 2 Structure of the document This document is structured as follows Public Final Version 1 0 Dated 07 10 2015 MODAClouds MOdel Driven Approach for design and execution of applications on multiple clouds Deliverable D3 5 3 e Section 1 contextualises the integration status and the relationship of the deliverable to earlier MODAClouds work e Section 2 overviews of the final integration prototype broken down to exploitable assets and their sub components e Section 3 describes the current status of the two MODAClouds integrated environments e Section 4 explains how to install RPMs and manually deploy and configure MODAClouds tools e Section 5 concludes the deliverable Public Final version 1 0 Dated 07 10 2015 Grant Agreement N FP7 318484 2 Overview of the Integration prototype The first version of the MODAClouds prototype represented the implementation of the plan described in D3 3 2 2 and was a working prototype bringing together the effort from various WPs using the Star Integration plan described in the proof of concept deliverable 6 The initial integration plan wa
10. Resource Repository MODAClouds IDE io ral A A Es ouds OpenSUSE 13 x based instance 4GB 2 CPU amp nae Space4Cloud OpenSUSE 13 x o dalla CloudML Application Data Collector Data Mapping Tada 4GB 2 CPU Ubuntu 12 04 based instance Filling the gap tool Model Repository D flexiant 4GB 2 CPU Ubuntu 12 04 4GB 2 CPU based instance Ubuntu 12 04 based instance j flexiant a aT 4GB 2 CPU flexiant a O Ubuntu 12 04 4GB 2 CPU PSA based instance Ubuntu 12 04 based instance i Monitoring Platform flexlant Support Platform Monitoring M Se erate Ubuntu 12 04 Queue Services OpenSUSE 13 x based instance Knowledge Base 4GB 2 CPU Ubuntu 12 04 ifi based instance jCloud Flexiant mOSAIC PaaS Unified connectors connectors connectors connectors Figure 4 FCO Final integrated testbed 3 2 IeAT Testbed The testbed offered by IeAT has been upgraded to the Eucalyptus version 4 5 of the cloud stack The upgrade benefits from the new Amazon EC2 S3 compatibility API update that makes the entire cloud API offered by Eucalyptus fully compatible with Amazons API Also the new version improves the starting process of the virtual machines by lowering the bootstrap process time of cloud images Moreover the new user interface has been upgraded by allowing the users to conduct all the management tasks through the UI without the need of using command line tools for managing the cloud resources At the hardware
11. Siemens Program and System Engineering BOC Information Systems GMBH Flexiant Limited ATOS Spain S A CA Technologies Development Spain S A Published MODAClouds documents Deliverable D3 5 3 Italy Norway Romania United Kingdom France Romania Austria United Kingdom Spain Spain These documents are all available from the project website located at http www modaclouds eu Public Final version 1 0 Dated 07 10 2015 Grant Agreement N FP7 318484 Contents 1 INTRODUCTION sisi epee aa aani 5 1 1 CONFEXAND OB ECRVES tr et dd det Ie 5 1 2 STRUCTURE OF THE DOCUMENT aiui rinaire da dt eS 5 2 OVERVIEW OF THE INTEGRATION PROTOTYPE oieee aadete bakaa care 7 2 1 VENUES AELOUD Suso illo a tocadas 7 2 2 CREATORA CLOUDS costando de cndud 8 FUNCIONA MOCOS AAA AAA AAA AAA AAA areea RS Inaa 11 E A O Mendes aaa on aan 11 A O A A es eee Senne eee ene ee eer ne nT ne 11 EINT eaten N dian seston a espa el Tandems V nlite tana A I tas 11 CIOUAME diia AAA AAA A ANNA AAA AAN A 11 RESQUECE IROPOSILONY ias 12 TCOC IOC LOOD RE EN 12 2 3 FANE GI ZERO D D nation 12 AD DGD ECTS 4 CLOUG S tn 12 Data Migrati Watt AA AAA AS A ARO 13 Load Balancer CON CEM A A OSO 13 UN O err ten Rar nT 13 AN y OLOR SE EXC IS ERG ey PIERO o A I O O me eT tern ee 13 MOSIMNATO A di lao 14 MOS POCKO0e DUNA ad A AAA ii 14 Tower AOUS ae eh ah a o Agee o E 14 PID FELIS COT Be rah isis A A OE ua iho AENA ai ve A aha A 16 DCO SOS A A ann tat ete A In A 16
12. component bridges the gap between the QoS model component provided by WPS and Modelio low level APIs It extends the Modelio IDE with commands that allow the QoS and monitoring rules to be validated and with mappings that allow monitoring rules to be generated automatically from QoS constraints Creator 4Clouds Prerequisites MODACloudML IDE requires the following pieces of software e JDK 1 6 or higher from http www java com e Modelio 3 1 2 open source from here Along with the following Modelio modules e Creator 4Clouds v 1 x y from http forge modelio org projects creator 4clouds files e ResourceModel v 1 x y from http forge modelio org projects creator 4clouds files e Persistent Profile v 3 0 02 Available by default on your Modelio installation e Java Designer v 3 0 01 Available by default on your Modelio installation Finally a MODAClouds runtime instance should be running somewhere so that monitoring rules and application can be deployed In this tutorial we will refer to this instance as localhost Installing Creator 4Clouds Adding modules from the Modelio modules catalogue Modelio modules are complementary components each of which provides specific services tailored to a particular modelling need Modelio provides a number of modules all of which exploit a model for a specialized need for example documentation or Java code generation When a module is installed it provides specific menus icons and specialize
13. consortium has ably delivered a working solution that fulfills the original objectives and MODAClouds goals The various individual components have achieved tighter integration to each other where apt and the monitoring and runtime platform finally integrated to the cloud infrastructure of Flexiant and IEAT converting mere testbeds and stand alone components to service offerings with a transparent back end This was achieved under the coordination of Task 3 3 with the IEAT and Flexiant infrastructure also used to develop the MODAClouds Supporting Services 5 The final phase has delivered an innovative MODAClouds framework as envisaged in the DoW and progressed the first iteration of D3 5 1 6 from proof of concept through to a working and usable cloud service offering The deliverable continues the new naming scheme of MODAClouds components adopted and repackaged to be more marketable together with a distinct branding The repackaged service offerings that make up the integrated solution can be summarised as e Venues 4Clouds consisting of Decision Support System DSS DSS User Interface Data collectors Risk Analysis Engine e Creator 4Clouds consisting of Functional Modelling Tool Space 4Clouds LINE CloudML Resource Repository e Energizer 4Clouds consisting of the follow sub packages o Tower 4Clouds consisting of Monitoring Manager DDA Data Collector Factory QoS Models Metrics Observer Metrics Explorer Knowledge Base Data
14. deleting resources present in Haproxy Moreover it also supports defining the load balancing policy as well as the weights Starting load balancing service can also be triggered remotely Object Store Code https github com i1eat mosaic object store Documentation https wiki volution ro Mosaic Projects ObjectStore Documentation Deliverable D6 5 3 License Apache 2 0 The Object store provides an alternative to the more traditional locally stored configuration files it can also be used by deployed services to store small pieces of data Artifact Repository Code https github com ieat mosaic artifact repository Documentation https w1k1 1eat ro SilviuPanica Projects mOSAIC ArtifactRepository Deliverable D6 5 3 License Apache 2 0 13 Public Final Version 1 0 Dated 07 10 2015 MODAClouds MOdel Driven Approach for design and execution of applications on multiple clouds Deliverable D3 5 3 This component stores MODAClouds artifacts such as deployment recipes Maven artifacts or software packages It provides a simple REST compliant API for managing the artifacts mOS Image Code https github com ieat mosaic mos image builder Deliverable D6 5 3 Documentation https github com i1eat mosaic mos image builder License Apache 2 0 This component builds a custom lightweight version of a cloud operating system based on OpenSUSE 13 1 mOS Image represents the core operating system of the mOS Platform mOS Package
15. deployment of the application at CPSM level ER model POLIMI Describes application data models at CCIM level Processing IMPERIAL Allows users to provide statistical information Extension on execution times Functionality2 Tier POLIMI Lists business operations implemented by each application tier Extension Multi Cloud POLIMI Defines partitioning of cloud resources in multi clouds Extension XML POLIMI QoS constraints to be respected at runtime Constraint Extension Resource XML POLIMI Assignment of resources to cloud providers Container Extension Usage Model POLIMI Usage model 24h distribution Palladio Allocation EMF Palladio Allocation of components to resources Model Palladio Repository EMF Palladio Description of high level application services Model Palladio Resource EMF Palladio Description of deployment infrastructure Environment Model Palladio System Model Palladio Description of high level service instances 10 Public Final version 1 0 Dated 07 10 2015 Grant Agreement N FP7 318484 Palladio Usage Model Palladio Description of usage scenarios Agreement Id ATOS SLA Id Table 5 Exchange formats Functional Modelling Tool Code http dev modaclouds eu svn modaclouds WP4 InternalDocs IDE trunk Documentation http forge modelio org projects creator 4clouds wiki WikiDeliverable Deliverable D4 3 3 License Apache 2 License The Functional Modelling Tool is implemented as a modu
16. gt Generate lt lt flow gt gt lt lt floa gt gt Validate meS MonitoringRulds xml Extension Resource Container xml E 4 we R SPACE4 Clouds Extension Resource Container xml gt gt PalladioResourceEnvionrment emf Venues4 Clouds lt lt flows gt PalladioAllocation emf PalladioRepositoty emf PalladioResourceEnvionrment emf Extension Resource Container xml PalladioSystem emf PalladioUsageModel emf Extension Resource Container xml Functional Modelling Tool Extension Usage Model xml Extension MultiCloud xml Extension QoS constraints cnn ProcessingTimeExtension xml lt lt flow gt gt gt Resource Database Resources Description sql CLoudML4 Clouds DeploymentModel json Agreement ld lt lt flow gt PalladioRepositoty emf PalladioUsageModel emf lt lt flow gt gt Agreement ld Energizer4Cloud ae d MonitoringRules xml lt lt flow gt gt Extension QoS constraints xml MonitoringRules xml lt lt flow gt gt l PalladioRepositoty emf y Functionality2Tier xml Agreement ld SLA Manager Tower4Clouds Filling the gap ADDApter4 Cloud Figure 1 Integration map File format Stakeholder Monitoring rules XML POLIMI Describes what should be monitored on cloud deployed resources and how to react to deviations Deployment model SINTEF Describes
17. is responsible for providing a monitoring platform that keeps track of the performance of the application and evaluates the QoS constraints defined at design time in Creator 4Clouds Space 4Clouds asset manages the lifecycle of the application shows the status of the multi cloud application and its execution environment to the operators and is able to perform certain reconfiguration actions depending on the metrics provided by Tower 4Clouds 2 1 Venues 4Clouds Venues 4Clouds is the decision support system of MODAClouds solution This component allows the users to express their requirements processes these requirements via risk based analysis and presents a list of sets of cloud services that are most suitable to meet those requirements It relies on generic data gathering mechanisms to collect the information about the characteristics of the cloud marketplace and systematic processing of users requirements into concrete and comparable characteristics of the cloud services Exploitable Module Component Integrated or External Asset service Dss UI Risk Analysis eos Data collectors Dss data Integrated Venues 4Clouds ia A ENNEN import DSS Data collectors Dss data Integrated save Table 1 Venues4Clouds Components It consists of User Interface in order to gather different sets of requirements data collectors and a risk analysis engine User Interface guides the user though the process of gathering different sets of requireme
18. 2 5 23 0 23 5 24 0 Iterations BackEnd_CPU_Processor FrontEnd_CPU_Processor Utilization Utilization 0 010 0 009 0 008 0 007 0 006 gt 0 005 0 004 0 003 0 002 0 001 0 000 gt gt AAA gt 7 e gt 00 05 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 10 0 10 5 11 0 115 12 0 125 13 0 13 5 14 0 14 5 15 0 15 5 16 0 16 5 17 0 17 5 18 0 18 5 19 0 19 5 20 0 20 5 21 0 21 5 22 0 22 5 23 0 23 5 24 0 Iterations BackEnd_CPU_Processor FrontEnd_CPU_Processor Response Times Response Time 0 0 A eee ee esees eaeaeaeaeaaee ee eee eee 00 05 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 10 0 105 11 0 115 12 0 125 13 0 135 14 0 14 5 15 0 15 5 16 0 16 5 17 0 17 5 18 0 18 5 19 0 19 5 20 0 20 5 21 0 21 5 22 0 22 5 23 0 23 5 24 0 Iterations query InvokeExternalService quickadd addcartbulk checkoutoptions checkLogin main processorder orderhistory orderstatus logout Figure 15 42 Public Final version 1 0 Dated 07 10 2015 MODAClouds MOdel Driven Approach for design and execution of applications on multiple Clouds Deliverable D3 5 2 Optimization trace ViolatedConstraints 5 6 7 8 9 6 7 8 9 Iterations Iterations Iterations Tier1VMs Tier2VMs Current Solutions Best solution violatedConstraints Figure 16 Public Final Version 1 0 Dated 31 03
19. 5 3 DeploymentScript Deploy MonitoringRules on the Executing platform Monitoring Rules C Users malmeida AppData Local Temp ConstellationSimple2706052321229416223 tt Monitoring Platform http localhost 8170 3 Deploy Monitoring rules Publish monitoring rules dialog 2 Set the address of the monitoring platform then click on the Deploy Monitoring rules button A 2 Space 4Clouds Installation and Usage Guide Features System PerformAnce and Cost Evaluation on Cloud is a tool for the specification assessment and optimisation of QoS characteristic of cloud applications Models of the application defined in Modelio and exported in the Palladio Component Model formalism with accompanying extensions are evaluated in order to assess both performance and cost of the modelled solution The tool is built on top of the Palladio Bench modelling environment but it differs significantly from Palladio since it enriches the modelling capabilities allowing more expressiveness in the definition of the resource environment and the specification of the workload SPACE4Cloud implements state of the art metaheuristic techniques to effectively and efficiently explore the space of possible alternative configurations For each configuration involved in the search process the tool is also capable of evaluating the overall operative cost However it makes use of tool LINE for the performance evaluation More information on the tool can be found in
20. Figure 9 then select as root directory the downloaded one and click on Finish a 2 General E Archive File 1 Existing Projects into Workspace Figure Y 38 Public Final version 1 0 Dated 07 10 2015 MODAClouds MOdel Driven Approach for design and execution of applications on multiple Clouds Deliverable D3 5 2 Either by copying the files or by importing the example the project should look like the one in Figure 10 Is Package Explorer 2 prrrrmmmssrrrrrrrrrrs de default allocation default allocation_diagram A default repository 8 default repository_diagram 4 default resourceenvironment default resourceenvironment_diagram A default system 3 default system_diagram A default usagemodel 2 default usagemodel_diagram LINE properties OfBiz Constraint xml resource_model_extension_OfBiz xml E usage_model_extensionOfBiz xml Figure 10 The following table briefly describe the files contained in the example project and their relation to SPACE4Cloud File Name Default allocation Default allocation_diagram Default repository_diagram Default resourceenvironment Default resourceenvironment_ diagram Visual representation of the resource environment model Default system Default system_diagram Default usagemodel Default usagemodel_diagram LINE properties Configuration file for the LINE performance evaluator NEEDED ONLY IF USING LINE OfBiz Constraint xml Extension fil
21. Grant Agreement N FP7 318484 MODAC louds Title Authors Editor Reviewers Identifier Nature Version Date Status Diss level Executive Summary MODA Clouds Integrated Solution Final Version Daniel Pop Gabriel Iuhasz IeAT Marcos Almeida Anthonin Abherv Softeam Michele Ciavotta Polimi Darren Whigham Flex Jacek Dominiak CA Nicolas Ferry SINTEF Rasha Osman Wang Weikun Pooyan Jamshidi IMP Craig Sheridan Flexi Danilo ARDAGNA Polimi Nicolas FERRY SINTEF Deliverable D3 5 3 Prototype 1 07 10 2015 Final Public This document deliverable D3 5 3 describes the technologies and environment that represents the final delivery of the MODAClouds integrated solution deliverable D3 5 3 It documents the implementation of the architecture of the MODAClouds solution detailed in D3 3 2 which defined the necessary interfaces to integrate the various components Copyright 2015 by the MODA Clouds consortium All rights reserved The research leading to these results has received funding from the European Community s Seventh Framework Programme FP7 2007 2013 under grant agreement n 318484 MODAClouds MODAClouds MOdel Driven Approach for design and execution of applications on multiple clouds Members of the MODAClouds consortium Politecnico di Milano StiftelsenSintef Institutul E Austria Timisoara Imperial College of Science Technology and Medicine SOFTEAM
22. M level to optimize deployment cost and performance At runtime the Creator 4Clouds sends a CloudML deployment model to the Models Runtime engine which in turn enacts the deployment In addition it sends a high level description of the application based on the monitoring rules and QoS constraints and a high level description of the deployment to the SLA manager which can in turn provide an SLA id that will be exploited by the Models Runtime engine to enforce the SLA Creator 4Clouds can also provide an ER model which will then transform it to specialized data models and be in charge of their deployment Finally during modelling work the IDE uses the QoS models component to generate monitoring rules from QoS constraints and to validate both of them Creator 4Clouds provides Tower 4Clouds with the monitoring rules and a description of the operations exposed by the components of the application to be monitored Figure 1 summarize the interactions between Creator4 Clouds and the other MODAClouds tools whilst Table 5 describes the different exchange formats Public Final Version 1 0 Dated 07 10 2015 MODAClouds MOdel Driven Approach for design and execution of applications on multiple clouds Deliverable D3 5 3 Data Mapping Component lt floae gt ER model xml Creator4 Clouds QoS Models Extension QoS constraints xmi Extension QoS constraints xml MonitoringRules XM cefow gt gt lt lt flow gt
23. OK on the warning and proceed with the installation A restart of eclipse is suggested after the installation has been completed Usage The entry point to the SPACE4Cloud tool in the Palladio Bench environment is the cloudy shaped icon that appears in the toolbar as shown in Figure 7 iar Run the SPACE4Cloud tool Ctrl 7 Figure 7 Public Final Version 1 0 Dated 31 03 2015 37 MODAClouds MOdel Driven Approach for design and execution of applications on multiple clouds Deliverable D3 5 3 Before starting the tool a project containing the required files has to be created or imported in the workspace In order to do that the user can make a new project by using the File gt New gt Project menu in eclipse and then selecting the template for a simple empty project as shown in Figure 8 Then import in the newly created project the needed files Select a wizard Create a Java project Wizards type filter text e Java Project from Existing Ant Buildfile Wz Plug in Project 4 gt General gt E CVS gt gt Eclipse Modeling Framework gt E Java gt gt Palladio Modeling gt 5 Plug in Development gt gt Xtend gt 25 Examples Figure 8 Another option 1S to download the sample project from ftp home de1 polim1 it outgoing Giovanni Paolo Gibilisco space4cloudExample and import it using the File gt Import menu Select the Existing Projects into Workspace as shown in
24. are Select a site or enter the location of a site y Work with type or select a site y Figure 3 Public Final Version 1 0 Dated 31 03 2015 35 MODAClouds MOdel Driven Approach for design and execution of applications on multiple clouds Deliverable D3 5 3 Insert the desired name e g SPACE4Cloud and use as location as shown in Figure 4 ftp home dei polim1 it outgoing Giovanni Paolo Gibilisco space4cloud Name SPACE4Cloud Location olimi it outgoing Giovanni Paolo Gibilisco space4cloud o Figure 4 Back in the Instal windows uncheck the Group Items by category option in order to show the available features select both the Space4Cloud and Palladio LINE Patch features and click Next as shown in Figure 5 Available Software Check the items that you wish to install Work with SPACE4Cloud ftp home dei polimi it outgoing Giovanni Paolo Gibilisco space4cloud Find more software by working with the Available Software Sites preferences type filter text Name Version G gt Palladio LINE Patch 1 0 0 201406091434 G Space4Cloud 1 0 0 201406091434 Deselect All Lp eee Details Show only the latest versions of available software Hide items that are already installed _ Group items by category What is already installed _ Show only software applicable to target environment Contact all update sites during install to find required software Figu
25. cations that make use of Glassifish 4 0 Application Server As part of ADDapters 4Clouds the CPIM library exposes APIs for access to the following Cloud services offered by the most popular PaaS platforms The platform is offered to give other providers the opportunity to create a connector and become eligible for being used within the MODAClouds platform 12 Public Final version 1 0 Dated 07 10 2015 Grant Agreement N FP7 318484 Component Integrated or External Data Migration and Data Migration and External Synchronization Synchronization bm p on Artefact Repository Table 6 ADDapters4Clouds Components Data Migration Code https github com deib polimi hegira components https github com deib polimi hegira api Documentation https github com deib polimi hegira components wiki https github com deib polimi hegira api wiki Deliverable D6 5 3 License Apache 2 0 This component supports data migration within two classes of NoSQL 1 e graph and columnar For columnar database it also supports synchronization between different replicas It can be started manually by a system operator or automatically by Models Runtime Load Balancer Controller Code https github com ieat modaclouds loadbalancer controller Documentation https github com 1eat modaclouds loadbalancer controller usage Deliverable D6 5 3 License Apache 2 0 This component is a RESTFul API for load balancer Haproxy It allows adding editing and
26. d annotations To add or remove a module from the modules catalogue the Add a module to this catalogue and Remove module from the catalogue buttons are used p Open the Configuration Modules catalogue command 2 In the Modules catalogue window click on Add a module to the catalog and use the file browser to select the modules jmdac files 3 Click on Close button when your catalogue is up to date Creating a new project To create a new Modelio project that is going to contain the MODACloudML model 26 Public Final version 1 0 Dated 07 10 2015 MODAClouds MOdel Driven Approach for design and execution of applications on multiple Clouds Deliverable D3 5 2 Save project Close project Import a project Switch workspace EJ Exit Project name Training System This project will manage the information system usedto manage all the training courses provided by the university IT departement Click on File New project Enter the name of the project Enter the description of the project Click on Create to create and open the project a ak g Adding a module to a project Project configuration Project properties Project s modules Modules installed in the project Expand modules catalog to add modules to the project Enable Scope Name 7 User D Modeler Module 81 10 9015 Started Project configuration Project properties Information Work models Libraries Modules Audit
27. deliverables D5 2 1 and D5 4 1 of the MODAClouds project Requirements In order to install and run the SPACE4Cloud tool the requirements listed in the table are needed Where to get it simulator com tools download LONS documentation 1 http www modaclouds eu wp content uploads 2012 09 MODAClouds_D5 2 1_MODACloudMLOoSAbstractionsAndPredicti onModelsSpecificationInitialVersion pdf 2 http www modaclouds eu wp content uploads 2012 09 MODAClouds_D5 4 1_PredictionAndCostAssessmentlToolProofOf Concept pdf 34 Public Final version 1 0 Dated 07 10 2015 MODAClouds MOdel Driven Approach for design and execution of applications on multiple Clouds Deliverable D3 5 2 LINE https code google com p line SPACE4Cloud can use either LONS or LINE as performance engines so at least one of the two has to be installed in the system Installation The installation of SPACE4Cloud over Palladio Bench 3 5 is performed using the install new software feature of eclipse It can be accessed as shown in Figure 2 by the menus Help gt Install New Software G Welcome d 2 Help Contents Search Dynamic Help Key Assist Ctrl Shift L Tips and Tricks Cheat Sheets Eclipse Marketplace Check for Updates Install New Software About Eclipse Figure 2 In the Install windows click the add a new update site by clicking the Add button as shown in Figure 3 Available Softw
28. e and Cost Evaluation on Cloud is a tool for specification assessment and optimisation of QoS characteristics for Cloud applications It allows users to describe software architectures by means of specific models that include Cloud specific attributes The tool can be used either to evaluate the cost and performance of a fully described solution application and Cloud configuration or to find a suitable even multi Cloud configuration that minimises the application running cost while meeting QoS requirements LINE Code svn svn code sf net p linesolver code trunk releases v07 Documentation http line solver sourceforge net releases Deliverable D6 5 3 License BSD License The LINE layered queueing network LON analyser interfaces with the Space 4Clouds tool in order to evaluate the performance of the candidate solutions evaluated by Space 4Clouds Space 4Clouds generates LON models and passes them to LINE which provides estimates for Response Time and CPU Utilization related to application components LINE can also be used as a standalone LQN analyser CloudML Code https github com SINTEF 9012 cloudml Documentation https github com SINTEF 9012 cloudml wiki 11 Public Final Version 1 0 Dated 07 10 2015 MODAClouds MOdel Driven Approach for design and execution of applications on multiple clouds Deliverable D3 5 3 Deliverable D4 2 2 License LGPLv3 CloudML is a domain specific language for specifying dep
29. e containing the constraints modeled by the user NEEDED Resource model extension OfBiz xml Extension file containing the mapping of PCM resource container to cloud resources NEEDED Usage model extensionOfBiz xml Extension file containing the 24 hour profile of expected incoming workload NEEDED In order to run SPACE4Cloud click on the cloudy icon of Figure 7 A pop up window will ask the user about the desired functionality currently the supported functionalities are Assessment and Optimization The Robustness analysis is in an experimental feature that is still unstable Both Features will ask the user to load the necessary Palladio models Figure 11 choose the desired solver Figure 12 and load the Usage Model Resource Environment and Constraint extensions Figure 13 Public Final Version 1 0 Dated 31 03 2015 39 MODAClouds MOdel Driven Approach for design and execution of applications on multiple clouds Deliverable D3 5 3 Resource Model Figure 11 Which Solver do you want to use rons me simucom Cancer Figure 12 Public Final version 1 0 Dated 07 10 2015 40 MODAClouds MOdel Driven Approach for design and execution of applications on multiple Clouds Deliverable D3 5 2 oi lala C space4cloud E OfBiz Constraint xml E resource model extension_OfBiz xml Y usage_model_extensionOfBiz xml File Name lusage_model_extensionOfBiz xml Files of Type XML files Fig
30. he CCIM model then select Creator4 Clouds gt Model transformations gt Generate CPIM deployment model The generated deployment model should include a type level model only and should look as follows a CPIM Model a E Deployment model a Generated Deployment alternative 4 Type level deployment model Fl SampleCloudNode a A E AResource a B El BResource ES Type level deployment diagram 2 Click on each element in the explorer and then use the element view to edit its properties illustrated below There you can define the high level aspects of your deployment ports connections OS and generic deployment scripts to be used during deployment Public Final Version 1 0 Dated 31 03 2015 29 MODAClouds MOdel Driven Approach for design and execution of applications on multiple clouds Deliverable D3 5 3 a Outline Fu Audit 9 Diagrams Z Link Editor E Script ap Element 23 E Notes and constrai gt Symbol El Persistance Zo Zy M Si Sun Modeler Module downloadCommand El Profil de persistance E Resource installCommand confiqureCommand startCommand stopCommand uploadCommand resourceExecuteLocally resourceRequiredCredentials Element view at CPIM type level 3 When you are done generate a instance level CPIM model To do so right click on the deployment alternative then select Creator4 Clouds gt Model transformations gt Generate new instance level model from type level model
31. ies The main policy is weighted round robin and the weights are derived at runtime based on demand estimations on the backend applications Models Runtime and Cloud Bursting Code https github com SINTEF 9012 cloudml Documentation https github com SINTEF 9012 cloudml wiki Deliverable D6 5 3 License LGPLv3 The Models Runtime component is responsible for enacting the provisioning and deployment of multi cloud applications as well as for adaptation actions such as scaling a certain tier of a cloud application This component is also able to migrate the application or part of it to a different cloud Self Adaptation Tester Code https github com imperial modaclouds query modaclouds mdload Documentation https github com imperial modaclouds modaclouds db retriever wiki Deliverable D6 5 3 License New BSD The Self Adaption Tester is a workload generation tool known as MDload with the purpose of simulating a set of users to generate requests to a web application in an automated fashion Auto scaling Reasoner Code https github com deib polimi modaclouds autoscalingReasoner Documentation http www modaclouds eu wp content uploads 2012 09 MODAClouds D6 5 3 RunTimeEnvironmentFinalRelease pdf Deliverable D6 5 3 License Apache 2 0 License This component determines and demands to the Models Runtime engine the adaptation actions scaling in and out that modify the application deployment in response to changes in the run
32. is responsible for Collect monitoring data from monitorable resources and send it to the data analyzers RDF History DB Code https github com deib polimi tower4clouds Documentation http de1b polimi github 10 tower4clouds docs Deliverable D6 3 1 License Apache License 2 Stores monitoring data and knowledge base history in RDF form collected at run time for offline analysis tools developed in WPS SpaceOps 4Clouds Space 4Clouds is a collection of tools designed to enable a cloud application to self adapt automatically modifying its deployment in order to meet predefined objectives and or satisfy predefined constraints whenever a change in the environment happens Software systems have historically been built as according to an open loop structure The evaluation of the system to be capabilities to meet QoS constraints is performed at design time Once the application is deployed any failure in guaranteeing the required service level would require human intervention which usually implies costly and time consuming operations and a decrease in revenue due to system low performance To cope with this issue we implemented a set of tools that provides out of the box and independently from the cloud provider a set of adaptive actions based on the feedback control loop which is very common in control theory and has already plenty of applications in controlled dynamic systems The adaptation mechanism in MODAClouds involves the defin
33. ition at design time and execution run time of policies Such policies can be activated in several ways they can be triggered by a monitoring rule registered in 16 Public Final version 1 0 Dated 07 10 2015 Grant Agreement N FP7 318484 the monitoring platform reactive policies as well as by an internal timer scheduled policies A Policy defines a protocol to follow to take decisions about adaptation actions Policies specify the way the application adapts to changes in the environment The action can therefore be described as a set of modifications of the deployment model or the load balancer Currently the Space 4Clouds consists of four major components Load Balancer Reasoner Models Runtime and Cloud Bursting Self Adaptation Tester and Auto scaling Reasoner Exploitable Module Component Integrated Asset ond Batancor Rewsoner Ines Load Balancer Reasoner odpi a Models Runtime and Cloud Integrated Bursting Space 4Clouds Space 4Clouds Self Adaptation Teste Auto scaling Reasoner Integrated MAR Table 11 Spae4Clouds Components Load Balancer Reasoner Code https github com imperial modaclouds modaclouds load balancer reasoner Documentation https github com imperial modaclouds modaclouds load balancer reasoner wiki Deliverable D6 5 3 License New BSD The Load Balancer Reasoner is a component for dispatching requests from end users to application servers following certain load balancing polic
34. itis oa 32 MA oie AAA II TR Tee mre nee Ue ee E eine ener eae ny eer evr 33 Public Final Version 1 0 Dated 07 10 2015 MODAClouds MOdel Driven Approach for design and execution of applications on multiple clouds Deliverable D3 5 3 Deploy rules A 2 SPACE 4CLOUDS INSTALLATION AND USAGE GUIDE cut abevbsbensuleschdelasceectaucs Public Final version 1 0 Dated 07 10 2015 Grant Agreement N FP7 318484 1 Introduction 1 1 Context and objectives This document provides a follow up to the MODAClouds platform integration plan deliverables D3 5 1 and D3 5 2 The deliverable is an output of work conducted during both Task 3 3 and 3 4 within WP3 This is also a companion document to D3 4 2 which reports on the integration of various MODAClouds components and is also due for submission at M36 This third and last integration phase at M36 concludes the 3 planned phases with the first deliverable released at M18 and the second at M30 This final phase sees MODAClouds software and tools running and hosted on the Flexiant and IEAT infrastructures and progresses and finalises the work delivered at M18 and M30 This final integration phase provides the third and last MODAClouds 1 integrated platform with new improvements and developments relative to the first proof of concept version and initial integrated versions The deliverable will show how the MODAClouds prototypes have progressed to an integrated solution as described in Task 3 4 and how the
35. le for the Modelio open source modelling tool It provides a graphical interface reverse engineering mechanisms and model checking to support the design of MODACloudML models at the CCIM CPIM and CPSM levels Besides the code for the functional modelling tool it contains the necessary code to connect to other design and runtime tools Communications to other tools are implemented by means of textual files generations and exchange CPIM library Code https github com deib polimi modaclouds cpim library Documentation https github com deib polimi modaclouds cpim library blob master README md Deliverable D4 4 1 D4 4 2 License Apache 2 License The library introduces a software abstraction layer between applications and cloud services by exposing Vendor Independent API In this way application code does not need to be rewritten in case of cloud services change Cloud services supported by the library are relational databases NoSQL databases message and task queues Blob storages mail services and Memcache service Moreover the library enables the Hegira 4Clouds component to seamlessly perform fault tolerant data migration and synchronization between heterogeneous NoSQL databases Space 4Clouds Code https github com deib polimi modaclouds space4cloud Documentation https github com deib polimi modaclouds space4cloud modaclouds space4cloud Deliverable D5 4 2 License Apache 2 0 License Space 4Clouds System Performanc
36. level see Figure 5 the testbed has new compute blades added to support a greater number of virtual machines available for the users Also the speed of the storage infrastructure has been increased from 4Gbps fibre channel technology to 40Gbps Infiniband With this upgrade virtual machines images are now transferred faster from the images repository to the targeted hosting server The upgraded testbed now consists of 8 blade servers with 64 cores and 80GB of RAM memory Regarding storage capabilities the cloud environment is linked to a 5TB SAN storage system MODAClouds components that are hosted on this testbed have been fully tested and are working as expected 20 Public Final version 1 0 Dated 07 10 2015 MODAClouds MOdel Driven Approach for design and execution of applications on multiple Clouds Deliverable D3 5 2 leAT Testbed Eucalyptus 4 5 STORAGE CONTROLLER SC WORLD CLUSTER CONTROLLER DEDICATED rr CC STORAGE SYSTEM CLOUD CONTROLLER CLC NODE CONTROLLER CONTROLLER CONTROLLER CONTROLLER CONTROLLER CONTROLLER NC NC NC NC NC NC VM Support VM Support VM Support VM Support VM Support VM Support ppor KVM KVM KVM KVM KVM KVM PP Hardware Hardware Hardware Hardware Hardware 8 CPUs 8 CPUs 8 CPUs 8 CPUs 8 CPUs 8 CPUs 10GB RAM 10GB RAM 10GB RAM 10GB RAM 10GB RAM 10GB RAM private connection Figure 5 IeAT Final Integrated Testbed 4 MODAClouds runtime platform build instructions
37. louds is a rich execution platform for multi cloud applications developed with the MODAClouds Creator 4Clouds Energizer 4Clouds offers solutions to common runtime problems such as service discovery packaging configuration deployment monitoring self adaptation and data synchronisation for multi cloud applications The platform is divided into two sub platforms the monitoring platform namely Tower 4Clouds and the execution platform that comprises two exploitable assets namely ADDapters 4Clouds and Space 4Clouds The following sections describe these three exploitable assets ADDapters 4Clouds ADDapters 4Clouds is the IaaS and PaaS unified layer responsible for connecting MODAClouds multi cloud Platform to several underlying cloud service providers at IaaS and PaaS levels By creating an abstraction layer exposing a Vendor Independent API to add different underlying providers ADDapter 4Clouds offers multi cloud agility interoperability and bursting among several of the most important IaaS and PaaS providers It offers a set of portable open source support services that can be deployed on any IaaS cloud to help package and deploy multi cloud applications From this IaaS deployment these support services can be used to support MODAClouds applications running either on laaS or PaaS Support services may be themselves distributed across multi clouds Moreover Support services have been extended in order to allow the deployment of local appli
38. loyment models that captures the deployable software artefacts the middleware required to execute them and the cloud resources providing the necessary computational resources CloudML also embed an engine to enact the deployment described in the models or can push them to the Models Runtime engine Resource Repository Model http forge modelio org projects creator 4clouds files Documentation http forge modelio org projects creator 4clouds wiki Wiki Deliverable D4 3 3 License Apache 2 0 License A MODACloudML model representing the most important cloud providers along with their cloud services and resource characteristics Its elements are integrated into application CPSM level models created in the Functional Modelling Tool Feedback Loop Code https github com imperial modaclouds modaclouds fg analyzer Documentation https github com imperial modaclouds modaclouds fg analyzer wiki Deliverable D5 3 2 License BSD 3 clause Feedback Loop is a tool for continuous parameterization of performance models It provides accurate estimates to parametrise the design time Quality of Service QoS models developed in Space 4Clouds Feedback loop aims at obtaining these estimates based on monitoring data collected at runtime once the application is deployed on the cloud It implements techniques for the estimation with different monitoring data 2 3 Energizer4Clouds MODAClouds Runtime Environment renamed as MODAClouds Energizer 4C
39. ls specification Final version D5 2 2 15 http www haproxy org 24 Public Final version 1 0 Dated 07 10 2015 MODAClouds MOdel Driven Approach for design and execution of applications on multiple Clouds Deliverable D3 5 2 A MODAClouds installation and Usage Details of how to arrive at a running MODAClouds platform can be found here https github com modaclouds modaclouds integrated platform and likewise details on running the components can be found here https github com modaclouds modaclouds integrated platform running the components There are also installation and usage guides for individual components and as an example Creator 4Clouds and Space 4Clouds are provided in the appendices below A 1 Creator 4Clouds Installation and Usage Guide Introduction Due to the advanced nature of the Creator 4Clouds development we are able to provide granular guidance on the usage of the tool The Creator 4Clouds is the piece of software that allows users to design cloud applications mostly independently of the cloud provider The IDE realizes the model driven engineering MDE approach leveraged by MODAClouds by means of the MODACloudML modelling language MODACloudML allows the users of the IDE to design an application in three levels of abstraction at the CCIM Cloud Computing Independent Model level the high level set of services that compose the application is defined along with its data model architecture and business and Q
40. nd the specified parameters a window will show its progress The window show in Figure 16 is divided into three sub figures The leftmost shows for each tier the amount of virtual machines used by the candidate solutions The number on the y axis refer to the sum of all the machines used during the 24 hours The central image shows the cost of the candidate in red and of the best solution found by the algorithm in blue per day in dollars The rightmost window shows the number of constraints that are violated by the candidate solution When the optimization process has finished an xml representation of the optimized solution can be found in the space4cloud folder of the project in the file solution xml Public Final Version 1 0 Dated 31 03 2015 41 MODAClouds MOdel Driven Approach for design and execution of applications on multiple clouds Deliverable D3 5 3 Configuration File config OptEngine properties Load Max Memory Size 10 Max Iterations 10 Max Feasibility Iterations 10 usan y Error in loading the configuration from file Figure 14 Number of VMs VmCount 100 75 gt 50 25 0 _ _ gt _ _ a _ _ __ _ _ _ _ _ _ _ Q _ _ _ _ _ _ _ _ 0 __ a 00 05 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 10 0 105 11 0 115 12 0 125 13 0 135 14 0 14 5 15 0 15 5 16 0 16 5 17 0 175 18 0 18 5 19 0 19 5 20 0 20 5 21 0 21 5 22 0 2
41. nd helps the user to identify the exact characteristics of the provider Data collectors Dss data import Dss data save Code https github com CA Labs dss data import Documentation README file Deliverable D2 3 2 License Apache Version 2 0 Acquires the data from multiple heterogeneous sources and transforms to common format that can be fed directly into DSS data store The module is designed as a standalone CLI based application and can be implemented across different types of uses not necessary connected to the Cloud Service Provider scenarios It can consume any type of web or local structured data source Risk Analysis Engine Code https github com CA Labs DSS Documentation D2 3 2 Deliverable D 2 1 2 License Apache Version 2 0 As an internal part of the DSS User Interface it allows the user to specify the risks and the values of the risks that can be faced by the deployment in question Given these inputs it creates risk matrix assesses if the risk needs mitigation and proposes the treatments for the risks that require mitigation User if free to allow the system to decide what is the level of risk mitigation needed in order to satisfy the requirements if decided otherwise the risk matrix mitigation values can be refined to match the exact required characteristics of a cloud service 2 2 Creator 4Clouds Creator 4Clouds offers a graphical modelling environment built on top of a set of targeted UML extensions that all
42. nts technical as well as business oriented assesses them and helps to define a matrix of potential risks with associated likelihood and consequence Furthermore 1t proposes potential treatments to mitigate the risks In addition User Interface provides different sets of visualizations to help the user comprehend underlying complexities in the cloud services arena Data collectors are sophisticated mechanisms to acquire process and feed the data from multiple sources like API s websites etc into the DSS Risks analysis engine takes input in the form of business and 7 Public Final Version 1 0 Dated 07 10 2015 MODAClouds MOdel Driven Approach for design and execution of applications on multiple clouds Deliverable D3 5 3 technical requirements from the user and conducts the risk analysis based approach to formulate set of cloud services characteristics that meet the user requirements ensuring risk mitigation The tree components of the DSS function together to provide an efficient service match making that recommends the users the set of services for required cloud types DSS User Interface Code https github com CA Labs DSS Documentation D 2 3 2 Deliverable D2 3 2 License Apache Version 2 0 Web based application which though the process of 6 steps guidance allows the user to express the requirements assess the risks and select the treatments Each step takes into account different dimension of cloud provider selection a
43. o add the generated rules to the instance level deployment diagram The result should look as follows 32 Public Final version 1 0 Dated 07 10 2015 MODAClouds MOdel Driven Approach for design and execution of applications on multiple Clouds Deliverable D3 5 2 lt lt Trace gt gt QoS Constraint VS CloudNodelnstance El SampleCloudNode On Demand Micro Instance Virtual Hardwares R Costs Cost Profiles lt lt monitors gt Metric Transformation Actions Monitoring rules in CPSM deployment diagram Deploy application l Deploy the application To do so right click on the CPSM deployment model then select Creator 4Clouds gt CloudML 4Clouds gt Publish application Deploy Application on Cloud Computing platform Based on CloudScript Tool JSON Deployment kc Users malmeida modelio workspace MODAClouds_C4C_Webinar data CloudScript Generated_GeneratedDeploymentalternative json Remote Execution Platform Runtime Platform ws 127 0 0 1 8080 Deploy application dialog 2 Click in Start deployment Deploy rules 1 Deploy the monitoring rules To do so right click on the deployment model then select Creator 4Clouds gt Tower 4Clouds gt Publish monitoring rules The following dialog should appear Public Final Version 1 0 Dated 31 03 2015 33 MODAClouds MOdel Driven Approach for design and execution of applications on multiple clouds Deliverable D3
44. o info modaclouds eu to obtain it e admin can use sudo without password e there is a developer account if you want to use it if not skip steps related to developer To switch user to developer sudo su developer You are advised to e change root admin and developer passwords e upload your public keys o ssh copy 1d admin IP o ssh copy 1d developer IP First Git has to be installed from admin user account sudo zypper install git From the account you would like to use git clone https github com modaclouds modaclouds integrated platform sudo modaclouds integrated platform sbin install platform sh lt vm hostname gt The last command adds the mOS repository to the system updates some system files installs the components and configures all MODAClouds services e g create database in MySQL for SLA Service IMPORTANT It modifies etc HOSTNAME and a line in etc hosts Please check the values are right especially if you had to run the script several times Avoid hostnames with underscore _ or dash characters 22 Public Final version 1 0 Dated 07 10 2015 MODAClouds MOdel Driven Approach for design and execution of applications on multiple Clouds Deliverable D3 5 2 Configuration After the installation a little configuration step is needed in each VM to know the IP addresses of the VMs and what services will run in each one This script must be run every time you want to redistribute the service
45. oS requirements At the CPIM Cloud enabled Provider Independent Model level a mapping of services onto cloud resources and the definition of generic deployment specification and QoS and rules allow the user to simulate and estimate the costs and quality of service of her design very early in the development Finally at the CPSM Cloud enabled Provider Specific Model level those concepts are translated to cloud specific concepts that can be directly translated into running and deployable code MODAClouds provides both design time and runtime tool to aid cloud application developers and application providers in creating cloud provider independent applications The IDE is the integration pivot of the MODAClouds design time tools It supports users in defining the application model at design time and in interacting with analysis and runtime tools provided by MODAClouds At the design group we include the Decision Support System DSS and SPACE 4Clouds At the runtime tools we include the Deployment and Provisioning Component CloudML the SLA Tool and the Monitoring Component Key Functions of Creator 4Clouds e The Design Manager component implements the MODACloudML metamodel as a UML profile on Modelio This is why all other components connect to it in order to access the MODACloudML model It also implements the necessary user interfaces features needed by a modelling tool commands for creating elements MODACloudML specific diagrams and
46. ows users to design multi clouds applications This set of extensions is collectively called MODACloudML MODACloudML provides methodological support for enabling developers to model multi cloud applications MODACloudML relies on three levels of abstractions e the Cloud enabled Computation Independent Models CCIM to describe an application as a set of services and interfaces between them e the Cloud Provider Independent Models CPIM to describe Cloud concerns related to the application in a Cloud agnostic way e g its deployment artefacts QoS constraints and external services to be reused e the Cloud Provider Specific Models CPSM to describe the Cloud concerns needed to deploy the application on a specific Cloud Designing a multi cloud application through the MODAClouds design time environment is a multi stage process First users specify through the Functional modelling component the application architecture and all its functional aspects as well as QoS requirements In the next stage on the basis of the high level assets that compose their application cloud application providers may exploit Venues 4Clouds in order to identify a set of Public Final version 1 0 Dated 07 10 2015 Grant Agreement N FP7 318484 relevant cloud providers Designers may then decide for instance to select a certain class of database services and certain kinds of computational resources This process is achieved by QoS engineers and supported by the
47. ptor is uploaded in the integration repository https github com modaclouds modaclouds integrated platform 5 Conclusion This document is the follow up to the latest integration report document D3 4 2 and the initial version of this deliverable D3 5 2 It reports the final implementation of the integration plan and a final release showcasing the MODAClouds solution as it stands at M36 This release and documentation show that the MODAClouds final integration and framework has been delivered on schedule as according to the detailed integration planning 6 References 1 http en wikipedia org wiki System_ integration 2 MODAClouds Integration Plan D3 3 2 3 MODAClouds architecture Final version D3 2 2 4 http www modaclouds eu software 5 MODAClouds Runtime Environment Initial Release D6 5 2 6 MODAClouds Integrated Solution Proof of Concept D3 5 1 7 MODAClouds Monitoring Platform Final Release D6 3 2 8 MODAClouds Precision and cost assessment tool Initial Version D5 4 2 9 MODAClouds Runtime Environment Proof of Concept D6 5 1 10 MODA Clouds Techniques for filling the gap between design time and runtime Initial version D5 3 1 11 MODACloudML IDE Final Version D4 3 3 12 Decision Making Toolkit Requirements and Architecture and Update on Business Methodology D2 3 2 13 https github com greese dasein cloud flexiant 14 MODACloudML QoS abstractions and prediction mode
48. rating new high level data and is configured by the Manager based on Monitoring Rules Statistical Data Analyzers can be used to compute more complex aggregations such as statistical estimates and forecast and are enabled automatically through Monitoring Rules A model representation of the monitored system is kept alive in the Knowledge Base which is an in memory RDF triple store embedded in the Data Analyzer The history of such model is stored in the RDF History DB automatically The Manager allows for attaching multiple observers to metrics generated by the Data Analyzer e The Metrics Explorer for showing metrics in a dashboard e The RDF History DB for saving the history of the metric e Any custom observer From the last deliverable D3 5 2 Tower4Cloud has been updated with the following features e Embed the knowledge base in the Data Analyzer to avoid performance issues by using Fuseki as the DB e The deployment model of the applications is updated by the Data Collectors e A new Web application is developed for installing and visualizing rules metrics observers and the model e Multiple serialization formats and transfer protocols have been implemented on the data analyzer in order to be able to attach different third party tools without additional adapters Graphite InfluxDB etc e Started to use Grafana http grafana org as visualization tool supported by Graphite 14 Public Final version 1 0 Dated 07 10 2015 Grant Ag
49. re 5 Eclipse will then compute the required actions in order to install patch the Palladio plugin that interacts with solution engines and install the SPACE4Cloud feature This operation might take a while Eclipse will then ask you to confirm the removal of the old version of the Palladio plugin and the installation of the patched one as shown in Figure 6 Proceed by clicking Next review the actions in the successive window then click Next again Review the license and accept them in the last window 36 Public Final version 1 0 Dated 07 10 2015 MODAClouds MOdel Driven Approach for design and execution of applications on multiple Clouds Deliverable D3 5 2 Install Remediation Page b The installation cannot be completed as requested Chose one of the following alternate solutions Keep my installation the same and modify the items being installed to be compatible e Update my installation to be compatible with the items being installed Show original error and build my own solution Solution Details Name Version Id a 3 Will be uninstalled Wp PCM Solver Feature de uka ipd sdq pcmsolver feature f a p Will be installed We Palladio LINE Patch 1 0 0 201406091434 it polimi modaclouds palladio pat Space4Cloud 1 0 0 201406091434 it polimi modaclouds space4cloud p p pa Figure 6 Eclipse will now perform the installation operations and warn the user that it is about to install unsigned content Click
50. re allowed To modify the hostname follow these steps sudo su HOSTNAME nodel echo SHOSTNAME gt etc HOSTNAME hostname F etc HOSTNAME The easiest way to start is using nodel as a hostname This way there is no need to change the configuration file You also must have an entry in etc hosts for that hostname associated to the IP address of the network interface Do not use localhost The line to add modify should be like 192 168 1 1 nodel localdomain nodel You only have to update your code and run the configuration script using the config file of your needs In the following we will use the configuration file using a single VM cd modaclouds integrated platform git pull Public Final Version 1 0 Dated 31 03 2015 23 MODAClouds MOdel Driven Approach for design and execution of applications on multiple clouds Deliverable D3 5 3 platform config modaclouds integrated platform modaclouds integrated platform lib config1 vm sh Check the content of bin common sh file It should have a VERSION variable greater or equal than 2 0 See the Configuration section for more details about the last step 4 3 Automatic deployment in mOSAIC The automatic deployment on mOSAIC is done according to the procedure described in deliverable D6 5 1 chapter 6 more precisely section 6 2 4 An application descriptor for MODAClouds Energizer 4Clouds deploys all the components of the environment on the mOSAIC platform The descri
51. reement N FP7 318484 The user manual of Tower 4Clouds is available in 14 Updated documentation is available at http deib polim1 github 10 tower4clouds docs An updated key components listof Tower4Clouds is reported in Table 8 along with the deliverables containing the detailed description and architecture of these components Exploitable Ass Module Component Integrated or External service Data Analyze RDF History DB Data Collector Java App Data Collector Integrated library T 4Clouds T 4Cloud l AO as Flexiant Nodes Data Integrated Collector Metris Explore Matlab Statistical Data Integrated Analyzers Java Statistical Data Integrated Analyzers Table 9 Tower4Clouds Components Monitoring SDAs SDA SDA weka Code https github com imperial modaclouds modaclouds sda weka Documentation https github com imperial modaclouds modaclouds sda weka wiki Deliverable D6 3 2 License GNU GPL SDA SDA matlab Code https github com imperial modaclouds modaclouds sda Documentation https github com imperial modaclouds Modaclouds SDA wiki Deliverable D6 3 2 License BSD 3 Clause Provide high level statistical aggregations such as prediction and estimation from collected metrics that can be reused by the Data Analyzer Data Analyzer Code https github com deib polim1 tower4clouds Documentation http de1b polimi github 10 tower4clouds docs Deliverable D6 3 2 License Apache License 2 Proces
52. s among VMs or a node s IP address changes platform config sh lt configfile gt You have several configuration files in modaclouds lib config files Select the one that better fits the distribution you need and modify it In addresses var enter a line for each node with the hostname and its address For each node in addresses var there must be an instances var prefixed with the hostname The file you use will be copied to modaclouds config sh so the next time you need to reconfigure you can simply use platform config sh modaclouds config sh NOTE The script knows the current node by reading the contents of etc HOSTNAME so it is important to match the names in the config file with the hostname Running the components You have to add the directory that contains the executable files of the modaclouds integrated platform to the PATH echo PATH PATH HOME modaclouds integrated platform bin gt gt HOME bashrc SHOME bashrc These are the basic commands platform start sh starts all services defined in platform env sh instance ids platform stop sh stop all services platform status sh check status of services platform service sh manages a single service start stop status The logs are stored in HOME Var log There is one log file per service Updating from v1 0 to v2 0 1 You may have a hostname not suitable for v2 that was not a problem in v1 remember no _ or a
53. s then documented in D3 5 2 This final integration deliverable is an extended version of the MODAClouds design time and run time platform which is hosted mainly on the Flexiant infrastructure and integrates the components described in the D3 2 2 architecture document 3 to the execution platform There are three major assets which have been integrated to the execution platform to constitute the overall MODAClouds framework Creator 4Clouds Venues4Clouds and Energizer4Clouds the latter composed of Tower4Clouds ADDapter 4Clouds and Space 4Clouds Creator 4Clouds asset is the Design Time Platform provided by the MODAClouds Toolbox It is a multi cloud modelling amp deployment integrated development environment which allows users to design cloud applications identify components and key functions of applications and define the desired QoS of the application Venues 4Clouds is a decision support system of the cloud application being modelled in Creator 4Clouds This component allows the users to express their requirements processes these requirements via risk based analysis and presents a list of sets of cloud services that are most suitable to meet those requirements Once the application has been modelled it can be deployed and managed in the clouds using Energizer 4Clouds ADDapters4Clouds asset is responsible for connecting the MODAClouds Platform to several underlying cloud service providers at IaaS and PaaS levels Tower4Clouds asset
54. ses at high speed the monitoring data coming from either the data collectors or SDAs It interprets filters and aggregates it based on monitoring rules 15 Public Final Version 1 0 Dated 07 10 2015 MODAClouds MOdel Driven Approach for design and execution of applications on multiple clouds Deliverable D3 5 3 Manager Code https g1thub com de1b polimi tower4clouds Documentation http de1b polimi github 10 tower4clouds docs Deliverable D6 3 2 License Apache License 2 It is responsible for installing monitoring rules configuring monitoring components and attaching external observers to requested metrics It provides both a Web GUI and a REST endpoint to interact with Tower 4Clouds Data Collectors Data Collector Java App DC Code https g1thub com de1b polimi tower4clouds Documentation http deib polimi github 10 tower4clouds docs data collectors java app dc html Deliverable D6 3 2 License Apache License 2 Data Collector Flexiant DC Code https g1thub com de1b polimi tower4clouds Documentation http de1b polimi github 10 tower4clouds docs data collectors flexiant dc html Deliverable No License Apache License 2 Data Collector Imperial DC Code https github com imperial modaclouds modaclouds data collectors Documentation https github com imperial modaclouds modaclouds data collectors wiki Deliverable D6 3 2 License BSD 3 Clause Update the knowledge base with the monitored resources it
55. t CCIM level 1 Create two services A and B To do so right click on the CCIM model then select Creator 4Clouds gt Elements gt Create Service 2 Define interface IA provided by A To do so right click on the Provided interfaces element in A then select Creator4 Clouds gt Elements gt Create provided interface 3 Define interface IA required by B To do so right click on the Required interfaces element in B then select Creator4 Clouds gt Elements gt Create required interface On the service assembly model under Orchestration open the Service assembly diagram 5 Use the Quick service instance to create instances of A and B and then connect them The final result should look as follows 28 Public Final version 1 0 Dated 07 10 2015 MODAClouds MOdel Driven Approach for design and execution of applications on multiple Clouds Deliverable D3 5 2 4 CCIM Model PP mmmmmmmmmmmmm rrmmmmrrmrm s Usage model 4 c A Service specification EJE Provided interfaces O IA ER Service interface diagram 4 Required interfaces Exchanged data Service implementation gt P lt She ARA Service specification Provided interfaces amp Required interfaces O IA ER Service interface Diagram Exchanged data Service implementation E Explorer view Service assembly diagram Model an application at CCPM level 1 Generate a CPIM model from the CCIM model To do so right click on t
56. t View for cloud provider accounts 30 Public Final version 1 0 Dated 07 10 2015 MODAClouds MOdel Driven Approach for design and execution of applications on multiple Clouds Deliverable D3 5 2 8 For each provider account in the elements view set its Base property to be the Cloud service from the provider 1t refers to Services can be found in the resources model see below 4 e ResourceModel 1 0 00 sa Q modacloud database 4 3 Amazon A id integer 0 A name string Amazon 4 laas Services 2 Elastic Compute Cloud EC2 2 Simple Storage Service 53 2 Elastic Block Storage EBS 4 PaaS Services a Relational Database Service RDS a DynamoDB Generic Google Microsoft DODC Flexiscale Resources model and cloud services 9 On the CPSM models you should define in which specific kind of cloud resource you want to deploy each node or external component of your application To do so open the Instance level deployment diagram move each element to the diagram and then create a ProviderWorkloadSpecfiication arrow between them To final result should look like this CloudNodelnstance E SampleCloudNode On Demand Micro Instance a Virtual Hardwares R Costs Cost Profiles Final instance level deployment diagram Model QoS constraints 1 Create a QoS constraint at CPSM level To do so right click on the QoS model under the CPSM model and then select Creator4 Clouds gt Elements gt
57. the necessary Ul for accessing the other components e The Model Transformation component allows the MODACloudML IDE to initialize a CCIM model from models of legacy components to initialize CPIM model from a CCIM and a CPSM model from a CPIM and to export the MODACloudML model to the MODAClouds tools e The Requirements Management component implements requirements modelling part of the CCIM modelling and the requirements traceability support that is part of the functional modelling tool It reuses the existing requirements modelling and traceability support in Modelio and extends it with cloud specific functionalities and diagrams e The Document Generator component allows the MODACloudML IDE to generate documents from a MODACloudML model It reuses the Document Publisher module of Modelio e The Audit Rules component validates the MODACloudML model being edited on the IDE by means of a rule based engine and a set of validation rules Rules are classified by severity level the WARNING rules indicate conditions that may hinder the applicability of some of the MODAClouds tools whereas ERROR rules indicate conditions that may hinder the applicability of all tools 47 rules were implemented in Java and delivered along with the IDE Public Final Version 1 0 Dated 31 03 2015 25 MODAClouds MOdel Driven Approach for design and execution of applications on multiple clouds Deliverable D3 5 3 e The Monitoring rules and QoS requirements management
58. time conditions Internally it implements a receding horizon approach and a Mixed Integer Linear Programming MILP model 17 Public Final Version 1 0 Dated 07 10 2015 MODAClouds MOdel Driven Approach for design and execution of applications on multiple clouds Deliverable D3 5 3 3 M36 Integrated Platform Testbeds A number of existing test beds have been used throughout the MODAClouds project for the development and integration of the components themselves and for the use of the MODAClouds solution by the case study providers Expansions and improvements have also been made to support the increasing needs of MODA Clouds as integration and experimentation progressed In the following subsections we list the most important used testbeds with a description of their underlying infrastructure 3 1 Flexiant FCO The testbed provided by Flexiant has been re architected to support a multi cluster environment and recently upgraded to support additional capacity Figures 2 and 3 detail the Flexiant Testbed Manager BU 16 cores CPU 8 RAM 128GB RAM 64GB Storage VLAN Control Panel VLAN Upstream VLAN Node Management VLAN Networks Key Figure 2 FCO Cluster 1 18 Public Final version 1 0 Dated 07 10 2015 Grant Agreement N FP7 318484 Upstream Router CPUS cores CPU 8 cores CPU 16 cores CPU 16 cores RAM 64GB RAM 64GB RAN GACR RAM 128GB RAM 128GB Storage VLAN Control Panel VLAN Upstream VLAN Node Management VLAN Ne
59. tworks Key Figure 3 FCO Cluster 2 To provide the MODAClouds project with a closer to industry testbed as possible a number of improvements have been made to the existing Flexiant testbed Currently the testbed has been expanded to include 4 new high performance compute nodes These nodes are each specked with 128 GB of RAM and 16 Cores which has resulted in an additional 512 GB of capacity an almost doubling of the previous capacity In addition to this capacity upgrade the backend storage for Cluster 1 has now been updated to use Ceph storage This has resulted in an increase of speed available to VM s as well as an increase in capacity to the total overall storage for the Cluster A further improvement has been the splitting of the FCO management box from the back end Database The Database has now been placed onto a new separate node This addition of a new node allows the database to be separate and not share the same resources as the management box This allows an increase in the total number of API requests as well and improved performance for both the Database and Web console With these upgrades completed the MODAClouds tools have been deployed across the testbed as detailed in the packaging procedures and installation instructions sections 19 Public Final Version 1 0 Dated 07 10 2015 MODAClouds MOdel Driven Approach for design and execution of applications on multiple clouds Deliverable D3 5 3 NN flexiant Application
60. u v2 packages modaclouds rpm or one of its mirrors The package json is a file used by the package builder script which provides all the details needed to generate mOS OpenSUSE compliant RPM packages such as general information external dependencies license or other resources 4 2 Manual Deployment This section explains how to deploy a MODAClouds Runtime Platform on VMs that have been already been provisioned producing a running platform with minimal configuration and installation effort A full set of scripts has been developed to aid the task allowing not only the installation and configuration of the platform but also to start services stop services and check their status This code is located in the following Github repository https github com modaclouds modaclouds integrated platform This method has been successfully used and tested by the case study providers using VMs and MODAClouds images provided by Flexiant The manual deployment can be achieved simply by provisioning VMs from the MODAClouds images then installing configuring and running the services as described below Installation These installation steps have to be followed for each VM They are suitable for any OpenSuse13 1 image There is such an image available in Flexiant called OpenSuse13 1_ v0 This image is the same image that can be found in OpenSuse repositories with some additional configurations e admin and root password are the same Send an email t
61. ure 13 If the user choose to perform the assessment after a short while a windows with the information about the utilization and the response time of resources and functionalities modeled in the application will appear Figure 15 If the user selected the Optimization a pop up will ask the user if he wants to automatically generate the initial solution since this feature 1s still experimental we will click on No and let the tool proceed with the solution described in the extension files Another Window will appear allowing the user to specify some parameters that are used by SPACE4Cloud to drive the optimization process as shown in Figure 14 The max iterations and max feasibility iterations heavily affect the time taken in the optimization process as they are used to evaluate the convergence of the algorithm the memory size 1s used to store candidate moves and can be increased 1f the type of available virtual machines is very big For most of the considered provided the default value of 10 1s sufficient The selection policy define how the algorithm select resources that have to be scaled out when a solution does not meet user defined constraints Experiments shows that the utilization policy is quite efficient in most situations but 1f no constraints on the Utilization of components have been provided other policies can perform better Since the optimization can take from few minutes to hours according to the complexity of the application a

D3.5.3 - MODAClouds

Contents

Download Pdf Manuals

Related Search

Related Contents