Lightwolf Team Blogs

Why Does the Public Cloud Look Like it Does?

Posted by Esmeralda Swartz on September 11, 2009 5:30pm EDT Comments (0)

Today, we are launching Lightwolf’s first blog. We will be updating this blog at least once a month–sooner, if we believe strongly in a topic.

IDC says cloud computing is “an opportunity in its infancy, but, even conservatively, poised to drive big marginal growth.” Burton Group adds that “the best organizations will use cloud computing’s unique business model, elasticity and scalability to streamline IT operations, offload lesser-value IT processes and focus on driving core business value.” Cory Doctorow writes “the main attraction of the cloud to investors and entrepreneurs is the idea of making money for you, on a recurring, perpetual basis, for something you currently get for a flat rate or for free without having to give up the money or privacy that cloud companies hope to leverage into fortunes.”

The pundits all agree on the future potential, but it is worth taking a look at what it will take to make the cloud a general purpose solution for Enterprise IT from its current boutique role. Today, one need only go out to Silicon Valley to realize that the hype far outweighs the reality. Whether you have a cloud solution or not, everyone wants to stake out a position in the next big thing. I am reminded of a time not so long ago of having the same experience with the number of companies springing up to take advantage of the next evolution of the Internet. Cloud computing is also being touted as the next evolution of the Internet and not just the Internet, but computing as well. No question, the potential is there, but we must realize that the build it and they will come mentality has been proven time and time again to be risky. In order for the cloud to become a general purpose solution for IT, consideration for the thousands of man years of best practices and Enterprise IT table stakes must be considered.

How did we arrive at today’s definition of the public cloud, anyway? The industry largely associates the public cloud with Amazon. The following represents my own observations and opinions on how Amazon shaped the public cloud definition.

The Amazon Cloud Case Study

The Amazon cloud was a result of many internal innovations dictated by their business model which drove Amazon’s incremental migration to the cloud. Its first storefront for books required innovation around massive scale of infrastructure to support their desired market penetration. The infrastructure was critical as Amazon’s responsiveness at the storefront was its competitive differentiation.

As other storefronts emerged (e.g. music and video) the Amazon innovation was around the path to closing the sale (Jukebox with 30 second samples for sampling before buying). This resulted in more bits to stream (Amazon player in the web browser) and requirements for data replication, caching, and geographic locality exploded. Amazon then needed to develop internal technology to support the storefront business ease-of-use (hit play with immediate response). The Amazon player and distributed key-value stores were infrastructure innovations.

Now that infrastructure was in place, the focus turned to something Amazon knew how to do really well, personalization of the user experience. Once again, innovation was required and this time it was around making suggestions based on buying patterns (correlated buys based on data mining). Amazon needed massive data analytics for access to buying patterns.

This is when it got really interesting; Amazon’s seasonality of business (e.g. the holiday rush) resulted in excess data center capacity throughout the rest of the year. In order to do cost recovery, Amazon then innovated around packaging capacity from the industry norm of long-term capacity commitments to rent-by-the-hour. They turned a liability into a new line of business. This meant development in operations support (virtualization, automated turn-up/tear-down, performance monitoring) was needed to support rent-by-the-hour. Amazon transferred their distributed systems innovations from their mainline business to its Elastic Compute (EC2) offering.

Once the new line of business was established, Amazon once again focused on ease-of-use. The focus turned to creating new applications for its cloud platform (easier to program) and enabling a broader range of applications.

Why was it important to go through the Amazon case study? Because once the industry associated the public cloud with the Amazon model, we saw attempts to pattern match other massive scale-outs as targets for the cloud (social networking, search, Apple Apps store, etc.). The back office also needs to scale so why can’t the back office use current massive scale-out solutions from the public cloud? The answer is simple. In order to achieve massive scale-out, other requirements were relaxed along the way. We believe that you have to start with Enterprise IT concepts and adapt these to the cloud:

• Security (including physical)
• Authentication, Authorization, Accounting (and Audit)
• ACID database, transactions
• Compatibility with existing code, design style
• High-availability, non-stop operation
• On-line processing
• Consistency/Coherency
• Thousands of man years of best practices

Post a Comment