Virtualization promises significant dollar savings. Since reducing cost is perhaps the biggest goal in today's corporate data centers, it's no wonder that virtualization efforts are accelerating. So far it's been easy to virtualize commodity servers that host less-critical utility applications. But after virtualizing all this "low-hanging fruit," IT now has to successfully virtualize higher-priority I/O intensive applications such as Exchange, SQL server, Oracle or SAP to improve its return on investment.

Performance is critical for these business applications, and ensuring production performance when they become virtualized is a difficult challenge. IT shops fooled by common virtualization management myths can't properly manage virtual enterprise infrastructure comprised of both clustered host servers and SAN-based storage.

Myth 1 Busted!: Virtualization Guarantees Performance
One of the biggest misconceptions with virtualization is that it ensures performance across the board for all client applications. It can be true that the average utility application gets a performance boost simply by the migration from an older, slower, and possibly quite small physical server into a virtual machine hosted on a newer, faster, larger server. By sharing large host servers, each virtual machine client can take advantage of excess host capacity that might be available at peak times. Enterprise virtual server features that load balance applications across a cluster of hosts can increase both server utilization and the resulting average server performance.

However, virtualized hosting simply does not guarantee performance to an application. Overloaded host servers deliver poor performance to all resident virtual clients. While a virtual machine can reserve resource capacity, if the total load on the host from all clients pushes the host into a poor performing state, all clients will still suffer. For example, if there are 10 virtual machines on a host each with a 10% reserve and all 10 applications in those virtual machines use 9% each, the physical host would be running at 90% utilization. Performance will be poor for every client even though each one is operating within its reserves. Simply increasing reserves to create an overhead buffer reduces the sharing opportunity and from a cost perspective is as bad as dedicating physical resources to non-virtualized applications.

Sharing a Resource Pool Does Not Guarantee Performance
Virtual machines also get a "fair share" of any remaining unreserved capacity. Unreserved capacity is doled out to applications in need according to a "share" setting. Even though shares change over time due to the dynamic competition from other applications, they can be analyzed and should be viewed as part of the virtual machine's total resource "entitlement" or guaranteed capacity. Of course, this total entitlement is still subject to the same performance problem as just described above for reserves.

Hungry applications can also share in any unused capacity reserved by other VMs (using the same share process as above). With a large number of applications, this scavenging can be quite effective on a statistically averaged time interval. However, any particular application at any single peak moment may not find any available excess capacity. Production applications that run beyond their total entitlements are at high performance risk since they are not guaranteed that extra resource at any point in the future.

Dynamic Resource Scheduling Does Not Guarantee Performance
Performance problems due to local resource competition can be alleviated with policy engines that load-balance virtual machines across a cluster of server hosts. In many commodity virtualization projects, these cluster-level features are relied on to "fix" performance hotspots automatically, albeit on the scheduling "intervals" of an hour or more. When handling production applications, there are major issues with relying on dynamic scheduling for performance management:

• A given cluster of servers used as a total resource pool may simply not be big enough for all clients at peak times.
• The scheduler may "thrash" and simply move a problem from one host to the next, hour after hour...
• The performance problem may not be solved by finding an application more server capacity, especially if it's really an I/O bottleneck.
• Even an hour of bad performance can be disastrous to business-critical applications.

Myth 2 Busted!: Virtualized IT Silos Can Be Managed in Isolation
A very common misconception is that virtualizing each IT infrastructure silo makes it easier to manage each as an independent utility. There is no doubt that application owners are finding it much easier to deal with a standardized "virtual machine" than with the complexities and details of widely varying physical hosting. Likewise, the virtual server admin maintains a certain independence from complex storage array details when simple virtual LUNs are delivered to him by the storage admin. Virtualization technology definitely simplifies the respective client-provider relationship and is fundamental to creating utility and cloud-based computing services.

However, the abstraction presented by virtualization represents a huge challenge for cross-domain system management. If you can't holistically see through layers of virtual "curtains," it becomes very hard to find unintentional resource contention. Within layered virtualization deployments the sheer number of managed objects and inter-relationships, not to mention the dynamic allocation and movement within each virtualization layer, means that statistically a larger number of underlying resources will experience unintended cross-contention between, and even within, complex applications.

Finding Bottlenecks Hidden Under Multiple Virtualization Layers
Finding performance bottlenecks buried under layers of cross-domain "wiring," e.g., applications to servers to storage to SANs to arrays to disks, requires cross-domain data path visibility and end-to-end contention analysis. IT silo or domain-specific tools are focused on operationally managing elements or resources within that domain. Many domain-focused system resource management tools can provide some logical or physical connection mapping to the next IT domain up or down, but usually not with over-arching performance analysis across the neighboring domains.

Optimization Requires Peeling Back the Virtual Curtains
Performance optimization is another special cross-domain challenge. Optimizing infrastructure within any IT domain, physical or virtual, requires:

• Upstream visibility: Intimate knowledge of client load profiles and service requirements
• Downstream visibility: Insight down into nested technology layers mapped to their physical performance characteristics
• Infrastructure performance model: Analysis of non-linear performance delivery under load and the relevant cost-performance trade-offs

Virtualization technologies make these optimization management tasks even harder due to the limited visibility upstream and downstream, dynamic resource sharing within resource pools, and the deliberate "fooling" of client management metrics (e.g., client O/S CPU utilization within a VM, and the client's view of remaining disk space under thin provisioning).

Myth 3 Busted!: Virtualized IT Can Be Managed Separately from Physical Infrastructure
It's unfortunate, but virtualization vendors tend to sell their clients a performance trap in the form of domain-specific management solutions. Sooner or later, a virtualizing enterprise data center will develop some contention through the sharing of their SAN-based storage between physically hosted and virtually hosted applications. Not only is it natural to migrate production applications with existing SAN storage configurations into virtual hosting, but production applications tend to leverage data accessed across the enterprise. These are some of the more troubling scenarios:

• Existing enterprise storage continues to be used in a bid to avoid disrupting a working application, affecting I/O performance, or spending inordinate effort in storage redesign
• Physical application volumes are moved into virtual server volumes, which are then mounted from the same enterprise storage to take advantage of performance and storage virtualization benefits

Problems creep into the first scenario due to the lack of management insight over both physical and virtual versions of the application, server, and storage performance side-by-side. In addition, advanced virtualization performance features like dynamic scheduling cannot be leveraged when retaining "direct" storage mapping.

In the second scenario, I/O characteristics can change considerably in the migration from physical volumes to virtual ones. Sharing, both intentional and unintentional, can cause hard-to-diagnose problems. Perhaps worst of all, existing management solutions may not be able to analyze both the virtual and physical clients together that share the same storage. Contention between a physical client and virtual client for storage can be invisible to existing IT management including:

• Storage management that might show that aggregate client performance is okay
• Virtual server management that might show that virtual host server performance is okay
• Physical server management that might show that physical host server performance is okay
• Application performance management that might show the physical applications are okay and due to the virtualized perspective within a virtual machine can't or don't model actual virtual application performance

Physical and Virtual Storage Clients Must Be Managed Together
The reality in most data centers is that some applications are in transition, some are in physical servers, some completely virtualized, and some with modules both physically and virtually hosted. For the applications that require SAN-based storage performance, it's critical to be able to manage end-to-end I/O performance during and after migration of any portion of the application into virtual hosting. To be completely successful when virtualizing I/O intensive applications, IT has to be able to de-conflict virtually hosted storage clients with physically hosted clients of the same enterprise storage services.

Expert Capacity Planning Is More Important Than Ever
As performance-sensitive applications migrate into virtual environments with pooled and shared resources, the capacity planning function becomes even more critical to IT success. The capacity planner needs to optimize the total investment in shared resources to achieve maximum cost savings, all while ensuring sufficient resources can be allocated to each critical application with business service levels. IT virtualization elevates planning from the server level to the more critical cluster level, and becomes increasingly important as more "eggs" are placed in the same basket.

As the previous sections have indicated, capacity planning is challenging for I/O intensive applications in environments with multiple layers of virtualization. Server resources need to be sized properly for performance while storage resources need to be sized for both disk space requirements and performance.

Thin provisioning features, really a way to virtualize disk space, are a good way to achieve storage cost efficiency. But like virtual server clusters, the requirement to plan enough actual storage space becomes even more critical as there is a risk that all applications can hit a hard wall all at once.

Don't Be Fooled by the Myths - Virtual Infrastructure Performance Requires a Virtualization Management Solution
Avoid falling for the myths by embracing virtual infrastructure performance management and capacity planning that enables you to:

• Ensure virtual infrastructure performance to meet service levels
• Get visibility across virtualization layers and IT silos to troubleshoot and prevent contention and bottlenecks
• Determine optimal load and plan for future growth of both virtual and physical resources