This is something I’ve been thinking about quite a bit lately, and in fact just a few days ago I was happy to learn that our paper, Disaster Recovery as a Cloud Service: Economic Benefits & Deployment Challenges, has been accepted into this year’s Workshop on Hot Topics in Cloud Computing (HotCloud 2010).  In our paper, we survey why we think cloud computing platforms are going to become increasingly popular for providing cheap disaster recovery services.

Clouds can be used to provide a variety of backup mechanisms ranging from cold replicas that are periodically synchronized up to hot standbys that are always in sync and can take over as soon as a failure is detected.  In practice, we think that a middle class of warm replicas is where the cloud can provide the greatest benefit.  A warm replica could be implemented as an EC2 VM that is not aways running, but whose disk (an EBS volume) is kept regularly up to date by a replication manager VM.  This replication manager can handle synchronizing the disk state for a large number of applications, but the customer will not have to pay for the active VM costs of those applications until a failure actually occurs and the VMs are booted up.

Check the paper for all the details, including a cost analysis of providing DR for various application types.

If you are interested, you can read the full version, or look through my slides.  It should make for absolutely thrilling bed time reading.

Here is the executive summary of what I’ve worked on:

Deployment

I start by looking at the deployment challenges of transitioning to a virtual environment and figuring out where to place VMs. This is an interesting area because virtualization can provide great benefits such as improved server consolidation, but also adds new challenges in the form of virtualization overheads.

MOVE (Modeling Overheads of Virtual Environments)

When you first consider transitioning from running applications natively to using virtual machines, it is important to understand how application resource requirements will change due to the overheads incurred by the virtualization layer. The MOVE project is designed to help predict these resource changes by building a regression model that relates the native and virtual platforms. This was work that I started during an internship at HP Labs in the summer of 2007, working with Lucy Cherkasova.

• Profiling and Modeling Resource Usage of Virtualized Applications. Timothy Wood, Ludmila Cherkasova, Kivanc Ozonat, and Prashant Shenoy. In proceedings of ACM International Conference on Middleware 2008.

Memory Buddies – Guiding VM placement with memory information

Once you know your resource requirements, you need to figure out where to put each of your virtual machines.  The Memory Buddies project tries to place virtual machines in order to maximize the amount of memory sharing that can be achieved — if VMs are running similar operating systems or applications, then the virtualization layer can share copies of these duplicated pages. In order to make this practical in a data center with many thousands of VMs, we propose an efficient fingerprinting technique that uses Bloom filters to quickly compare virtual machine memory contents.

• Memory Buddies: Exploiting Page Sharing for Smart Colocation in Virtualized Data Centers.  Timothy Wood, Gabriel Tarasuk-Levin, Prashant Shenoy, Peter Desnoyers, Emmanuel Cecchet, and Mark Corner. In proceedings of the International Conference on Virtual Execution Environments, VEE 2009.

Resource Management

Making data centers more efficient is a key concern throughout all of my work.  Virtualization’s greatest benefit comes in the promise of improved server utilization, leading to lower hardware costs and decreased energy consumption.

Sandpiper – automated VM loadbalancing

Alright, now we’ve figured out initial resource allocations and placements for all of our virtual machines, but those initial decisions may not be sufficient (or efficient) if an application’s workload changes over time. Sandpiper is a system which monitors the resource utilization and performance of a set of VMs and dynamically adjusts their resources or migrates them between hosts in order to prevent servers from becoming overloaded. This was the first project I worked on when I came to grad school, and now there are several commercial products out there doing similar things. We recently revised and extended this paper for a journal.

• Black-box and Gray-box Strategies for Virtual Machine Migration. Timothy Wood, Prashant Shenoy, Arun Venkataramani, and Mazin Yousif. Proceedings of the Fourth Symposium on Networked Systems Design and Implementation (NSDI), Cambridge, MA, April 2007.
• Sandpiper: Black-box and Gray-box Resource Management for Virtual Machines. Timothy Wood, Prashant Shenoy, Arun Venkataramani, and Mazin Yousif. To appear in Computer Networks Journal Special Issue on Virtualized Data Centers 2009. (Extended version of NSDI 07 paper)

Reliability

High performance systems are only useful if they are reliable. The remaining work for my thesis uses virtualization to decrease the cost of high availability and fault tolerance systems.

ZZ: Cheap Practical Byzantine Fault Tolerance

Byzantine Fault Tolerance is a way of providing very strong reliability guarantees, even in the face of malicious users or application components.  Unfortunately, BFT has a very high cost because each application request must be executed 2f+1 times in order to handle f simultaneous faults. In ZZ, we try to reduce this cost down to only f+1, by using an additional f sleeping VM replicas which are only woken up after a fault is detected.

• ZZ: Cheap Practical BFT Using Virtualization. Timothy Wood, Rahul Singh, Arun Venkataramani, and Prashant Shenoy. University of Massachusetts Technical Report TR14-08, 2008.

CloudNet: Wide Area Resource Management and Availability

My most recent work was started while at AT&T in Fall 2008, and looks at how VPNs can be combined with cloud computing platforms to make data center resources appear seamlessly connected to an enterprise’s existing infrastructure. We are further exploring this area to see how we can provide disaster recovery services so that if a data center becomes unavailable, the critical applications running within it can transparently fail over to servers at a different data center.

• The Case for Enterprise-ready Virtual Private Clouds. Timothy Wood, Alexandre Gerber, K.K. Ramakrishnan, and Jacobus van der Merwe. In proceedings of the Workshop on Hot Topics in Cloud Computing, HotCloud 2009.

Open Cirrus Cloud Computing Testbed: Federated Data Centers for Open Source Systems and Services Research

Roy Campbell, Indranil Gupta, Michael Heath, and Steven Y. Ko, University of Illinois at Urbana-Champaign; Michael Kozuch, Intel Research; Marcel Kunze, KIT, Germany; Thomas Kwan, Yahoo!; Kevin Lai, HP Labs; Hing Yan Lee, IDA, Singapore; Martha Lyons and Dejan Milojicic, HP Labs; David O’Hallaron, Intel Research; Yeng Chai Soh, IDA, Singapore

This is a very large (more than 10K nodes spread across 9 sites) testbed being setup by HP and others to study large scale cloud computing problems. They are focusing on computation provisioning issues, and can provide users with either full physical or virtual resources.

Nebulas: Using Distributed Voluntary Resources to Build Clouds

Abhishek Chandra and Jon Weissman, University of Minnesota

The idea here is to explore the potential for creating peer-to-peer style cloud computing platforms that uses resources provided by volunteers similar to something like SETI @home.  I like this idea a lot, but there have been many attempts at making things like volunteer based network file systems which never quite took off, and this seems even harder.  The difficulty will be determining what the basic platform that people are given access to is like (ie. can you run any app you want within some VM, or is it a specific platform you must develop your app against to make it work), and how do you make the resources shared by users not impact their own application performance.  People are pretty willing to share network bandwidth and disk space, but that is because those are generally over provisioned resources.  CPU is over-provisioned in a different way — most of the time desktop users use only a fraction of the power provided by their system, but when they do decied to go do something computation intensive, they expect it to respond quickly.  This also reminds me of the “transparent memory contribution” work done by Jim Cipar when he was still at UMass, since it had to deal with similar issues of volunteering resources in as transparent a way as possible.

The Case for Enterprise-Ready Virtual Private Clouds

Timothy Wood and Prashant Shenoy, University of Massachusetts Amherst; Alexandre Gerber, K.K. Ramakrishnan, and Jacobus Van der Merwe, AT&T Labs—Research

I thought this paper was really great, but maybe I’m biased since I wrote it.  I’ve written a separate blog post about my own work, but the gist is that current cloud computing platforms are insufficient for enterprise users, and we propose using network virtualization techniques to make seamless and secure connections between the cloud resources and enterprise sites.

ElasTraS: An Elastic Transactional Data Store in the Cloud

Sudipto Das, Divyakant Agrawal, and Amr El Abbadi, University of California, Santa Barbara

The idea here is that databases currently don’t scale well into the cloud.  Instead people are using simpler (but more easily scaled) key-value stores to keep track of data in the cloud.  This doesn’t work well because key stores don’t provide the transaction and consistency features of real databases. They propose Elastras – a scalable, trasactional data store based around the idea of partitioned databases. It wasn’t clear how difficult the problem of determining how to partition data is in the first place, as it tends to be application specific.

Reflective Control for an Elastic Cloud Application: An Automated Experiment Workbench

Azbayar Demberel, Jeff Chase, and Shivnath Babu, Duke University

The idea of reflection is to make an application change its behavior based on the available resources.  This could be based on energy or computation resources.  This lets you opportunistically exploit surplus resources, and to defer work during congestion. An example of a reflective application is a digital experiment (generally has large data sets, can be partitioned, does not have strong time requirements). Seems to me like this is useful for any batch processing style application.  The work focuses on figuring out how to determine the utility of running different experiments depending on what resources are available, which may be very difficult since the experiment design space can be huge. It seems to me that the idea of reflective applications is useful even at a more basic level to both let applications be aware of what resources are available and for service providers to know what applications desire.

Colocation Games and Their Application to Distributed Resource Management

Jorge Londoño, Azer Bestavros, and Shang-Hua Teng, Boston University

This paper explores the placement problem within data centers using game theory techniques. In general they find that a Nash Equilibrium will not be reached, but that in a restricted environments it will always converge.  I’ll be interested to look through their results more carefully to better understand how the potential for multiplexing resources in these environments can be reduced based on the self-interests of users.

Virtual Putty: Reshaping the Physical Footprint of Virtual Machines

Jason Sonnek and Abhishek Chandra, University of Minnesota

The idea here is that the physical footprint required by a VM can vary depending on its environment. For example, VMs colocated together may be able to share memory, or may require much fewer network resources if they can put on the same LAN. To exploit this, you need to estimate the “virtual” footprint of a VM that captures how its physical requirements can change depending on its environment.  The first challenge here is to efficiently capture this model — you will only be able to get a significant benefit from this kind of technique if it is being applied across a very large number of VMs (my memory sharing work suggests this as well). Second is the issue of determining how to deal with applications changing over time – memory and network communication patterns may change over time, so how often do you need to recompute the footprint?

Statistical Machine Learning Makes Automatic Control Practical for Internet Datacenters

Peter Bodík, Rean Griffith, Charles Sutton, Armando Fox, Michael Jordan, and David Patterson, University of California, Berkeley

The goal here is to model application performance and automate management online. Models are based on data gathered from the system as it is running, allowing it to be adapted as more data is produced. Has some automated techniques to detect phase shifts in application type that will require a new model. The problem with these systems is always a question of how well they can deal with data that is outside of their training data.  One of this system’s benefits is supposed to be that it doesn’t rely on training data produced from experimental setups, and instead builds the model on the fly as data is gathered. Bbut of course that may mean that the models are only really applicable for “normal” operating conditions, and that it will not be able to make reasonable predictions for what will happen after a load spike.

Other Hot Cloud Reports

I’ll add any other hot cloud blogs or reports as I find them (or comment below).

• NetworkWorld 5 Cool Cloud Computing Research Projects – sadly does not mention me… ;]

You can find a copy of our paper here, and my slides here. Or you can get a summary of our ideas below.

Here are the three key features we feel are lacking from existing cloud platforms:

Security: Enterprises need strong security guarantees about the isolation of both the computation and network resources they are getting from the cloud. Existing systems rely on firewall rules for security that must be configured on a per-VM basis. While firewalls are a very powerful form of access control, they are incredibly fine grain and need to be carefully configured. This is a especially a problem in highly dynamic (ie. cloud) environments where new VMs are often being created or moved between servers.

Transparency: Another problem with cloud computing is that the resources it gives you are completely separated from the systems an enterprise is already running within its data centers. This makes it difficult to deploy applications since you can’t get the abstraction of having your cloud resources seamlessly connected to your existing LANs within the enterprise.

Resource Flexibility: There are two issues here. First, existing cloud platforms grant users very limited control over the network resources connected to their VMs.  This means, for example, that it is impossible to do something like reserve a high bandwidth link between a pair of VMs, and certainly not between a VM and the enterprise site that is going to be accessing it. Secondly, cloud platforms are not as flexible as they should be: if you replicate a VM to increase the processing power of an application you need to deal with these security and transparency issues all over again.

To help provide these three features, we propose the idea of a Virtual Private Cloud, that uses VPNs to securely connect groups of VMs within a cloud data center back to the enterprise sites that will use them.  VPNs make it so that the cloud resources are only accessible by other members of the same VPN.  This is a much coarser grain access control mechanism than firewalls, but it is much cleaner and we use MPLS based VPNs that have the benefit of being both highly scalable for enterprises that may run many hundreds or thousands of VMs, and that require no endhost configuration on the VMs — the VPN is entirely setup at the routers at the cloud and enterprise sites.  Finally, there is the option of using layer 2 VPNs (a Virtual Private LAN Service) to bridge the cloud computing data center and enterprise networks, giving the abstraction that cloud resources are seamlessly connected to the enterprise’s own LAN.

We are building a system that will implement this sort of system, and are exploring how it can be used to simplify VM migration over the WAN and for providing high availability services capable of seamlessly failing an application over from one cloud data center to another.

Kevin Lawton (author of old-school virtualization tool Bochs), recently wrote an article (and followup) on how you could speed up VM migration in a data center by exploiting things like page duplication between the source and destination machines.  The idea is that a lot of VMs have common memory pages that you wouldn’t actually need to copy over.  He references some of my own work that looked at the amount of sharing that actually occurs between VMs, the first (but hopefully not last) time I’ve ever seen a link to one of my papers in a random blog post in my RSS feed!

The only problem I see has to do with detecting when pages are truly identical. When you do page sharing between VMs on a single host, you detect the similarity by producing a short (32 or 64 bit) hash for each page in memory. If you scan two pages and they produce the same hash, then those pages are very likely identical. I say “very likely“, because it could just be a hash collision, and you need to actually scan all the bits in each page before you can truly know that the two pages are identical.

When you do this on a single machine, it’s not too big a deal to scan two pages and compare their bits, but if you are trying to verify that pages on the source and destination of a migration are identical, it is a big problem. Obviously you can’t just copy the page over the network to do the comparison, since that is what you were trying to avoid in the first place. I guess the only solution is to use a longer hash value (thus reducing the chance for collisions) and really hope that you don’t have a malicious VM at the destination that is trying to corrupt your memory by purposefully creating memory pages that will collide with your content.

I think it is a neat idea that would generally work in practice, but you will need a pretty smart cache at each end to make sure your really keeping the pages consistent.

Another idea would be to use a “post copy” based approach that tries to get the VM started up on the destination machine as quickly as possible, deferring copying most memory pages until after it has already started.  You might be able to use this to quickly unload a host that is approaching the overload limit, although the migrated VMs may see a larger performance penalty because of how the migration technique works.

Cloud based services are also better for the environment.  If the average everyday computer can be reduced down to a basic thin client for accessing remote cloud services, that reduces the cost and energy usage of home devices.  The applications running in the cloud can in turn exploit massive degrees of multiplexing to reduce their own energy costs.  As it stands today, most peple’s computers are far more powerful than they really need to be, and they spend a lot of time sitting around idle.  You might as well give consumers devices that are as simple as possible.  Make them so simple that they don’t crash all the time.   Make it so they don’t require technically adept family members to service them a few times a year. Then people will spend less time being frustrated by their computers, and more time able to use the applications running on them.