How virtual servers make the cloud more elastic

As more companies begin to move their applications into the cloud, many are finding out that not all applications are created equally. By that, I mean not all applications are tailor-made to migrate into the cloud.

Part of the problem for enterprise organisations is the workload demand that certain applications require: either a large number of processing cores or larger amounts of memory than traditional cloud architectures can accommodate. While virtualisation technology is normally the answer to many cloud-related questions, this time virtualisation has to be used and thought of in a different way in order to meet the challenge.

When we typically talk about virtualisation, we do so in terms of partitioning or making many smaller virtual machines out of a single larger physical machine.

In doing so, we optimise the workload of a single physical machine, but what happens when you need to go the other way? What happens when you need to optimise the workload demand of an application that requires more processor cores or more memory than a single physical machine has to offer?

ScaleMP is one company providing such a technology. Its new virtualisation technology is being described in terms of virtualisation for aggregation rather than consolidation.

To find out more, I spent some time with Shai Fultheim, the founder and president of ScaleMP, and we talked about the differences between scaling up and scaling out with virtualisation.

InfoWorld: You and I were talking about different ways of viewing server virtualisation.  Can you explain these a bit more?

ScaleMP: The best analogy for the two ways to view server virtualisation is to think of how IT, for years, has viewed storage. There are two ways to purchase and implement storage: large storage arrays and smaller in-system, JBOD or NAS.

In the former, companies have partitioned large storage arrays for particular workloads, users, or departments. The single large array provides them an easier way to manage and distribute storage. In the latter, there have been storage management tools available to concatenate the distributed smaller arrays to appear as a large pool or single storage resource. This is exactly what is now occurring in server virtualisation.

Most IT professionals are familiar with server virtualisation for consolidation, which, like the first example in storage, consolidates workloads and increases the utilisation rates of a single x86 system. This type of virtualisation is focused on scaling out, or horizsontal scale, and provides compute resources to transactional or stateless applications that are the majority of IT data centre workloads.

The latter form of virtualisation, or what we call virtualisation for aggregation, provides the concatenation virtualisation for those workloads that require more processing power and more memory than one system can provide. In the traditional server world, this is called scaling systems up, or vertical scale. This capability takes numerous small servers and creates large virtual systems with tens or hundreds of CPUs and single system RAM in excess of 4TB.

InfoWorld: How then does virtualisation for aggregation work, and do you find it is best suited to specific environments or workloads?

ScaleMP: Server virtualisation for aggregation solves three fundamental problems facing high-performance IT today by being cost effective; offering simplified cluster installation and management; and scaling up aggregation and redistributing cloud resources for high performance or large memory workloads.

Today the largest virtual SMP is created from up to 16 discrete x86 server nodes. This creates a virtual SMP with 128 cores and 4TB of RAM. However, with the introduction of the new Nehalem EX 8-core processors, the size of the virtual SMP grows dramatically.

In addition, ScaleMP will shortly announce support for 32-node systems. Both of these advances mean that by the end of the year customers will be able to build low-cost SMPs with thousands of cores and close to a hundred terabytes of RAM.

On a per-core basis, these systems will be significantly lower cost than a proprietary SMP but will obviously be larger than any but the very largest supercomputers. The revolutionary thought in all this is that the cost of building the most powerful system configurations will drop dramatically, and any IT organisation with cloud infrastructure will be able to aggregate a supercomputer on the fly.