Virtualization is one of the key enablers of cloud computing. Virtualizing hardware, applications and consolidating them aims to reduce IT Infrastructure costs (which includes purchasing hardware and maintaining it), and allowing easier management of resources in the data center. The IaaS model of the cloud deals with provisioning of compute capacity, storage and networking resources. Infrastructure costs are reduced by virtualizing hardware, thereby avoiding under-utilization of resources. As a result, multiple applications can be virtualized over the same physical hardware ensuring optimal usage of resources. The other terms that we often hear w.r.t cloud computing are ‘on-demand’ and ‘elasticity’. These two terms go hand-in-hand; on-demand refers to a ‘pay-as-you-go’ model. You need to pay only for the resources you use, and elasticity refers to scaling up/down the resources at will. In the context of cloud computing, the virtualization lifecycle comprises a set of technical assessment activities which are governed by business and operational decisions. Technical assessment for virtualizing candidates revolves around meeting end-user Service Level Agreements (SLA’s), reducing IT costs, and designing an optimized data center. Every phase in the virtualization lifecycle for cloud computing is highly challenging with a wide variety of complex and open problems which are currently being tackled.

Analysis & Discovery: When migration from Physical environments to Virtualized environments (P2V), in-depth analysis of the virtualization candidates must be performed. This stage involves discovering the data center entities (servers, networks, storage devices), collecting utilization profile data of applications along the different dimensions (CPU, memory, network i/o, disk i/o). The main theme of P2V is to move applications from an under-utilized bare metal environment to a virtualized / hypervisor environment to enable optimal utilization of hardware. In addition to discovering the heavy artillery in the physical environment, it is important to assess the applications deployed on them. The OS characteristics, application performance footprints play a vital role in determining capacity in a virtualized environment. On completion of these assessments, capacity management models need to be developed for the virtual environments.

Implementing Capacity Models: Developing capacity models for a virtual environment is a tricky task since it is governed by other business and operational factors. Target SLA's (performance, availability), power consumption levels are to be considered along with the possible impacts (side effects) of virtualization (hardware normalization, hypervisor overheads, IO interference etc). The idea is to come up with a 'pre-VM placement' strategy which describes the 'footprints' of VM's. Capacity planning for virtualized data centers in the light of cloud computing has become a highly sought after topic.Determining capacity size of virtual machines apriori to migration becomes an extremely critical step. If done accurately it can result in optimal allocation and usage of resources, if over-cooked, can lead to resource wastage. Similarly if under-cooked, can result in poor performance and violation of SLAs. There are many useful P2V, V2V and capacity analysis tools that can help you achieve this, viz. Platespin Recon, Microsoft SCVMM, VMware P2V Assistant, to name a few. Also researchers are exploring intelligent ways of doing capacity sizing in Virtual environments.

VM Placement & Management: This is the most critical process from a data center administrator point of view. Academic and Industrial groups are grappling to identify 'best-fit' placement strategies to enable highly optimized virtual environments. This refers to the concept of 'packing' VMs appropriately. VMs need to be packed in such a way that perfomance of isolated (individual) VMs is not hampered due to interference, and to avoid fragmentation in the data center. The on-demand provisioning of virtual servers will eventually lead to a server sprawl, thus complicating management of virtual servers. Hence efficient techniques for placement and management hold key in having a greener and well maintained data center. Other issues may also involve cross data center migration, synchronization between different servers, enabling and managing hybrid clouds (a combination of public and private / in-house environments).

Thus, the virtualization life cycle poses many challenges in different areas. Leading Cloud providers and academicians are busy solving these problems, and we hope to see greener data centers soon !!

Autoperf currently works only on a Linux Operating System, so here we will install AutoPerf on Ubuntu 10.04 Lucid.

To begin with, on the Client machine where you would run the Master module, you need to have Java installed along with the CLASSPATH set to something like this in the /.bashrc file  ” export CLASSPATH=/home/aditya/Desktop/AutoPerf-Shrirang/AutoPerf-Master/code/jar/log4j-1.2.13.jar:/home/aditya/Desktop/AutoPerf-Shrirang/AutoPerf-Master/code/class “.

The AutoPerf-Master folder structure should look like this : 

The input.xml file is the one that we will be using with the Load Generator as the input file.  Also ensure that the XML file is consistent in terms of tags and parameters. The input.xml specifies the following :

- Transaction name ( i.e the operation / Web service to be invoked )

- Target URL address of the Web service

- No. of Concurrent users

- Think time in milli seconds

- IP address of the Server machine

- Port address at which the prof agent is running

A sample input.xml file looks like this :

Now, lets take a look at the Server Machine. To setup the Profiler agent on the Server, you must begin by installing the standard package and libraries of ‘gcc’ ( libgcc ) from the Synaptic manager,  otherwise you will run into errors while initializing the prof agent.

The folder structure for the Linux Profiler on the server should look like this :

Once the packages have been installed on both, the client and server side, you are ready to start load testing applications provided you have your webservices up and running.  Also, take care to see if the input.xml file is not missing any tags, else you would run into parsing errors.

To start load generator using the Master, you first need to initialize the prof agent at the server side. This can be done using the command ‘ sudo ./prof -d 2011′ . Execute this comment when you’re inside the LinuxProfiler directory as show above. Executing this command starts the prof agent at the port 2011. This now means that the agent will pick up any incoming requests at this port address, and will profile the Webservice which is hosted at some particular web address.

However, you need to start the Master component from the client machine using the following command ‘ sudo java Master input.xml’ . Executing this command will parse the input.xml and start the load generation ( load testing ) based on the parameters of the input.xml file ( no. of users etc ) .

Lets now take a look at the profiling of a sample Complex Addition web service deployed at the server side.

Make sure the prof is running at the server. This can be verified using the ‘ps’ command, and will look like this :

At the client side, you will start the load generator, and will see an output like this :

Once you get such an output , it means that the prof agent has captured all the performance metrics and sent the output to the Master module. When the ‘java Master input.xml’ command is issued at the client machine, the Webservice at the specified URL is invoked by 10 concurrent users, with a think time of 1000 msec. Therefore, the output generated is at that load level.

[1] AutoPerf : An Automated Load Generator and Performance Measurement tool for multi-tier software systems

Performance Models / Modeling

1. Performance Models for Virtualized Applications

2.Profiling and modeling resource usage of virtualized applications

3. Black-box performance models for virtualized web service applications

4. Probabilistic performance modeling of virtualized resource allocation

5. Automatic virtual machine configuration for database workloads

6. Towards Modeling & Analysis of Consolidated CMP Servers

7. Modeling Virtual Machine Performance

Provisioning

1. Autonomic virtual resource management for service hosting platforms

2. Virtual Putty

3. Efficient resource provisioning in compute clouds via VM multiplexing

4. On Dynamic Resource Provisioning for Consolidated Servers in Virtualized Data Centers

5. Resource Provisioning with Budget Constraints for Adaptive Applications in Cloud Environments

6. Utility Analysis of Internet Oriented Server Consolidation in VM BasedData Centers

Profiling / Interference

1. XenMon: QoS Monitoring and Performance Profiling Tool

2. An Analysis of Performance Interference Effects in Virtual Environments

3. VrtProf

As we all know by now, the motivation behind Virtualizing applications is to ‘do more with less’, increase ROI, reduce TCO, create a greener environment and so on, planning the size of virtual machines hosting these applications becomes a key aspect. Server consolidation is a means to achieve higher utilization of servers, which may be under-utilized in a dedicated physical environment. Placing multiple VM’s across a shared resource pool is governed by target SLA’s, optimizing power consumption, optimally sharing physical resources, workload type (database, web server)of the application. Sizing and managing capacity of these virtual entities becomes an important factor during the virtualization lifecycle in the context of cloud computing. Understanding issues of VM interference (cache interference, i/o interference), hypervisor overheads should be helpful in sizing VMs.

Analysis could be done for P2V, V2V migrations, thereby estimating VM size and adapting according to existing (current) bottlenecks and future trends. There are many useful P2V tools and capacity analyzers made available by vendors.

- PlateSpin Recon

- Microsoft SCVMM

- Oracle VM Manager

- VMware P2V Assistant

- HP Capacity Advisor

- Vkernel Capacity Optimization

In this article we just scratched the surface of  ’Capacity Planning for Virtual Environments’ . In the next part we shall see detailed aspects related to interference and performance of applications in a hypervised setup.  Here’s one of my favorite literature on capacity planning http://esj.com/Articles/2010/07/13/Capacity-Planning-Virtual-Environment.aspx?Page=1

David Linthcium’s blog

Cloud Switch

Software Design & Construction

Lanamark

TIBCO Silver

Thomas Bittman ( Gartner )

AMD Blog

VMblog

HostedFTP

ElasticVapor

CloudTweaks

The Subject and object are modelled as nodes, while the predicates are denoted by links in the graphed structure. Nodes are stored and efficiently reused when required. An RDF triple in the semantic store has a subject ( start node ), predicate (relationship), object ( end node ), which comprises a link. A new link is created on inserting a new triple and nodes are reused if similar nodes already exists.

New object types are defined to manage Semantic Data viz. SDO_RDF_TRIPLE and SDO_RDF_TRIPLE_S. The former stores the references to the data and the latter holds the actual data content. The nodes ( Subject, Object ) are stored in the RDF_NODE$ table, which can be further broken down into START_NODE_ID and END_NODE_ID. The RDF_LINKS$ table stores the record for the link whenever a new triple is inserted. Blank nodes may also be inserted as a part of any triple, which are stored in the RDF_BLANK_NODE$. An RDF model stores references to all the RDF data in the database and can be created by executing the sem_apis.create_sem_model procedure.

To get started with semantic data management on Windows XP, click “http://thesemanticway.wordpress.com/2009/01/04/configuring-semantic-web-technology-support-in-oracle-11g-release-1-on-windows-xp” to configure semantic web technology support in Oracle.

This article gives an overview of semantic data storage,however to get additional in-depth information on Semantic Data support in Oracle, here are some useful links:

http://download.oracle.com/docs/cd/B19306_01/appdev.102/b19307/sdo_rdf_concepts.htm

http://download.oracle.com/docs/cd/B28359_01/appdev.111/b28397/sdo_rdf_concepts.htm

References : RDF support in Oracle (http://www.oracle.com/technology/tech/semantic_technologies/pdf/semantic_tech_rdf_wp.pdf)

1. Install Oracle 11g and download the “JENADRV” patch from “http://www.oracle.com/technology/software/tech/semantic_technologies/files/jenadrv_patch111rdf.zip“.

2. Extract the JENADRV folder to some directory.
3. Goto Directory Your_Drive_Name:/>Oracle_Home/product/11.0.1.0/md/admin
4. Open SQLPLUS and connect as SYS user.
5. Type the following command at the SQL prompt

- SQL>@Your_Drive_Name:\>Oracle_Home\product\11.0.1.0\md\admin\catsem10i.sql;

This command restores Oracle 10 RDF data.

6. Now issue the following command at the SQL prompt

- SQL>@Your_Drive_Name:\>Oracle_Home\product\11.0.1.0\md\admin\catsem11i.sql;

This command installs 11 RDF, confirm with the snapshot given below to check if procedure has been executed successfully.

7. Now, alter the “mdsys” user by the following command

-SQL>ALTER USER mdsys ACCOUNT UNLOCKED IDENTIFIED BY mdsys;

8. Connect as user “mdsys” as show below

9. Now, apply the JENADRV patch to enable Semantic web Data support. Execute the following commands in the same sequence shown below

- SQL>@Your_Drive_Name:\>Extracted_Jenadrv_folder\sdordfh.sql;

-  SQL>@Your_Drive_Name:\>Extracted_Jenadrv_folder\sdordfxh.sql;

- SQL>@Your_Drive_Name:\>Extracted_Jenadrv_folder\sdordfa.sql;

-SQL>@Your_Drive_Name:\>Extracted_Jenadrv_folder\sdordfb.plb;

-  SQL>@Your_Drive_Name:\>Extracted_Jenadrv_folder\sdordfxb.plb;

- SQL>@Your_Drive_Name:\>Extracted_Jenadrv_folder\sdoseminfhb.plb;

- SQL>@Your_Drive_Name:\>Extracted_Jenadrv_folder\sdordfai.plb;

10. Connect sys as sysdba

11 . Create a new “rdf” tablespace by issuing the following command

CREATE TABLESPACE <<TABLESPACE_NAME>> DATAFILE ‘Your_Drive_Name:\>Oracle_Home\oradata\<SID>\<<TABLESPACE_NAME>>01.dbf’ SIZE 128M REUSE AUTOEXTEND ON NEXT 64M MAXSIZE UNLIMITED SEGMENT SPACE MANAGEMENT AUTO;

12. Similarly create a temporary tablespace

CREATE TEMPORARY TABLESPACE <<TEMP_TABLESPACE_NAME>> TEMPFILE ‘Your_Drive_Name:\>Oracle_Home\oradata\<SID>\<<TEMP_TABLESPACE_NAME>>01.dbf’ SIZE 128M REUSE AUTOEXTEND ON NEXT 32M MAXSIZE UNLIMITED

13. Now, create a new “testuser” and grant required privileges as shown in     the snapshot

14. Create a semantic network to enable semantic data management

15. Create table

16. Create a semantic model to enable a semantic data environment

Once this is done, you have enabled the semantic web technology support in Oracle 11g Release 1. Happy Semantic Data management !!

1. Goto http://projects.semwebcentral.org/frs/?group_id=37&release_id=192
2. Download “Code-Lib-Feature-1.1.zip” and “OWL-SEditor-1.1.zip”.
3. Create a folder on your local computer “OWL-S IDE”, copy and extract the downloaded files in the folder.
4. Start Eclipse
5. Click on “Help —-> Software Updates —-> Find and Install”

6. Select “Search for new features to install” and click “Next”.

7. Click “New local site”

8. Browse to the “Code-library feature” directory and select it.

9.  Select ok.

10. Select the checkbox next to the Code-lib feature and click Finish.

11. Select check box and click Next.

12. Accept agreement and click Next.

13. Click finish

14. Click Yes to restart eclipse.

15.  Follow the same procedure for “OWL-S Editor feature folder ” .

16.  Click Finish

17. Click yes to restart eclispe.

18.  Select “Help —-> Software Updates —-> Manage Configuration” to verify that the plugin has been installed properly.

19. Check if the contents in the ellipse appear.

20. Help —-> Help Contents.

21. Check if the contents in the ellipse appear.

Now, you will be able to create OWL-S descriptions in the OWL-S IDE.

Service ontologies supplements the WSDL interface, adding knowledge required to enable automatic discovery, invocation and composition of web services. The idea is to annotate web services, enabling the automation of the web service life cycle.

The existing conceptual models for describing services are OWL-S, WSMO, WSDL-S, SWSF, SAWSDL.

For example, the OWL-S service ontology is classified into three categories : profile, model, grounding as shown in the diagram. The service component is actually an instance of the service and is linked to the profile, model, grounding by different properties. The profile is an advertisement of what the service does i.e what the service offers in terms of functionality. It considers input, output, preconditions, effects ( IOPE ).

The input specifies the actual input required for invoking the web service, output specifies the actual output the client gets or expects. Preconditions indicates the conditions that need to be satisfied for the successful execution of the web service. The effect describes the state of the web service after its execution.

The service model describes how the service works in order to achieve its functionality. It describes atomic processes, composite processes and the message choreography involved in invoking the web service. Atomic processes are the ones, that undergo straight forward execution requiring standard input, whereas composite processes are the ones which involve combination of different services.

Service grounding illustrates as to how the service can be accessed.It describes the network protocols, data exchange formats, required to invoke the web service.

Like OWL-S, the other models also address the semantic nature of web service descriptions thereby making an effort to automate the web service life cycle.