What Is Best Suited For The Cloud

by Jay Judkowitz

This is the second in a series of four articles discussing infrastructure as a service (IaaS) clouds. The series started at basic level setting and we will now begin diving progressively deeper. The topics for the series are:

1. Cloud 101
    - What is cloud
    - What value should cloud provide
    - Public, private, and hybrid cloud
    - Starting on a cloud project
2. Application taxonomy, what belongs in the cloud, and why
3. What you should look for in cloud infrastructure software
4. Evaluating different approaches to cloud infrastructure software

Cloud is obviously a serious transformation for your datacenter, but that transformation does not need to be far off or futuristic. If you pick the right applications owned by the right users with the right needs and if you partner with the right cloud software provider, cloud and its many benefits are achievable today.

When first deploying a cloud, it is critical to choose the right applications to move to the cloud initially. Given that cloud is about allowing business units to manage their own computing needs, it follows that the ideal place for cloud is where the business units:

  • Expect self-service.
  • Are tolerant of incorporating provisioning logic into their day-to-day work.
  • Have variable compute needs for the tasks that generate the need for frequent provisioning and de-provisioning activities.
  • Have multiple tasks that cause context switching in the customers’ work, with different tasks acquiring and yielding compute resources over time.
  • Have workloads that, even at maximum interaction, do not cause complex resource contention issues on shared resources so that as much or as little can be deployed as necessary with little if any forethought or calculation.

If you break down the types of workloads in a datacenter, you get three major types:

  • Traditional, monolithic, and stateful client/server apps – things like Exchange Servers and traditional databases
  • Scale-out load balanced apps with disposable stateless instances
  • Batch type computing jobs that can be decomposed into small chunks of compute and storage and distributed across a pool – things like Hadoop, Monte Carlo simulations, business analytics, and media processing and conversion

For each class of application, there are dev/test deployments and production deployments. This gives us a simple six-way classification of what runs in a datacenter that we can use to select our cloud candidates.

The following chart shows which workloads are good for early cloud adoption and what should be dealt with later on in the process. Beneath the chart is some explanation and justification for this assessment.

(Green represents near-term opportunity and red represents something that should be sent to the cloud later on)

Scale-out Load Balanced Apps with Disposable Instances

This type of application does not rely on any one instance of software being able to grow to use tremendous amounts of compute resource. Rather it assumes that each instance can only do so much and that additional power will be supplied by adding more instances. For this sort of application to work the data and other application state must be driven out of the application itself to another location.This allows instances to be created and destroyed with no data loss or outage to the end user. The worst problem caused by failure of an instance is the need for the end user to retry their operation. When the operation is retried, it finds a live instance and completes without difficulty.

Generally, the application requires greater or lesser instances over time as load grows and shrinks. This creates a need for dynamism in the deployment with low turn-around time on provisioning operations. As load grows, the application must respond in a short period of time. Either the application administrator or some auto-scaling management system needs to be able to make the required changes without a ticket to the infrastructure team. When load is high, new instances must be spawned to handle the increased demands. When load drops, instances should be deleted to reclaim compute resources for more useful activity. Because the instances are stateless, without sufficient load that needs to be serviced, no instance has an inherent reason to remain persistently deployed.

Scale-out applications are also a great fit with the datacenter operations models required to manage a massive cloud.  Cloud scale datacenters need to assume that any piece of equipment can break at any time with the application being resilient and able to hide the failure from the end user. Scale-out applications accomplish this by spreading many instances across nodes, datacenters, or even geographies. This makes it much simpler for the infrastructure to be managed – failures become a capacity issue which can be managed in aggregate on a periodic basis, not end-user outage issues that need to be addressed immediately and individually.

More generally, this type of application is the way of the future in general. Due to current systems architectures, we are seeing more and more cheaper cores distributed across many commodity nodes rather than the massive scaling up of individual servers. The only way to really get applications to scale is to build for many smaller instances teaming up together. Some programming infrastructures take this to a logical conclusion. Node.js, a language increasingly used for scale-out web applications, refuses to give a program access to more than one core regardless of how many cores are in a system. Should the developer need more power, they need to increase the number of node.js application instances.

Scale-out applications and cloud are such a good match because they have the same goals – elastically, scale up and down in response to load using only the resources actually needed, distributing applications across hosts and sites to better tolerate systems outages without end user impact and better conformation to modern system architecture.

Batch Applications

Batch applications are decomposable to smaller compute and storage packages. They are good in both test/dev and production for similar reasons as the scale-out applications. As jobs are launched, the number of instances depends on how fine grained you can chunk the job. Given different sizes of data sets for separate runs, the number of instances needed for each run will vary. Therefore, statically provisioning instances doesn’t make sense. Furthermore, there are likely to be completely different applications in such an environment that will need to run at different times. Clouds are perfect for repurposing an infrastructure, ramping up one application while ramping down another without having to do a massive retooling of the physical infrastructure. Like with the last case, the infrastructure administrators should not decide how many and when to deploy instances. Activity needs to be driven by the business that is actually running the jobs and deriving benefit from the output.

Traditional IT Applications

Traditional IT applications, vertically scaled, stateful and monolithic are not good for clouds in their production deployments. These applications tend to be custom built for a particular load in mind, deployed once, expected to have each instance stay running persistently, and are only touched for upgrade. These types of applications are not a natural fit for the cloud paradigm. They do not accommodate dynamic scaling by simply adding more instances. As a result there is a reduced need for end-user self-service.  Also, since they are monolithic, any performance or availability problem must be addressed in the one or two instances that make up the application with a deep understanding of the impact of the underlying infrastructure. As a result, the burden of management remains on the infrastructure administrator and cannot shift the application administrator. This breaks the operational model of cloud where the datacenter administrator needs to be removed from the details of the running of applications.

However, traditional IT applications are very well suited to cloud when the service is development, integration, and testing of the applications. For this function, application owners need to deploy new instances of server software, update them, make new templates, try out their work and iterate. Furthermore, mass numbers of clients will need to be deployed for load and scale testing. The development and testing process generates the dynamism in workload resource needs as well as the requirement for self-service on the part of the IT developer that make a cloud an ideal solution.

Pick the Right Applications and Move to Cloud Today!

Too many times cloud is pitched as an evolutionary technology. In many cases, this is because the vendor making this pitch is already managing a legacy application stack for the customer and sees no reason for a radical shift.

Since these legacy applications do not accommodate elasticity and do not tolerate the more unpredictable availability of any single server that the cloud datacenter operations model implies, true clouds are limited in the benefits they can provide and cause a loss of SLA that is unacceptable to the end users of the legacy applications.

Of course, legacy applications will not go away any time soon and we acknowledge that it takes tremendous time and effort to move to a new programming paradigm. But, the technology is here today and the benefits have been made obvious – scale, resiliency, efficiency. The success stories of companies like Netflix and Zynga are well known. All that is needed is the will to move in that direction.

For enterprises and service providers that leverage modern applications development process, cloud is not an evolution at all – cloud is the best and most obvious way forward for development, testing and mass deployment of their applications.

Pick your target applications and get started today!

Cloud 101

by Jay Judkowitz

This is the first in a series of four articles discussing infrastructure as a service (IaaS) clouds. The articles will start with basic level setting and will dive progressively deeper as the series progresses. The topics for the series will be:

1. Cloud 101

  • What is cloud
  • What value should cloud provide
  • Public, private, and hybrid cloud
  • Starting on a cloud project

2. Application taxonomy, what belongs in the cloud, and why
3. What you should look for in cloud infrastructure software
4. Evaluating different approaches to cloud infrastructure software

What Is Cloud?

Cloud is fundamentally about creating a dynamic computing infrastructure that enables end users to service and manage themselves in a process that is frictionless and instantaneous, but also secure and controlled. Resources are allocated in a very fine-grained fashion and can be relinquished at any time. Usage and chargeback is measured per customer, per resource utilization, and per unit of time, not by physical piece of equipment.

Cloud is not an incremental step on top of virtualization. While virtualization is a key enabler of cloud, the motivation, focus, target applications, and evaluation criteria are very different.

For clouds to be useful and cost effective, they must also have the properties of being very scalable and smooth to manage at the infrastructure level. The idea is to remove as much as possible from the day-to-day operations from the datacenter IT team and to scale that group’s reach and efficiency. The primary responsibility of the datacenter managers should be scaling the cloud in response to growth in demand. They should be able to look at utilization in aggregate and plan and deploy space, power, networking, and servers in as much of a just in time manner as possible. They should no longer need to focus on project specific deployment activity – that is handled by a combination of the end users’ self-service activities and automated responses from the cloud itself.

Private clouds represent the transformation of an IT datacenter into a large self-service pool of resources for internal customers to use. Public clouds open up that service to external organizations to purchase. Hybrid cloud refers to when one organization uses a private cloud for some of its work and a public cloud for other work; and where there is continuity between the public and private cloud utilization. In hybrid clouds, identity, policy and user interface are common, thereby blurring the distinction between public and private to the end user. This series discusses cloud in general with public, private and hybrid being implementation decisions for specific projects. We will introduce distinctions between these cloud types only where necessary.

Cloud Business Value

The end result of cloud to the business is two-fold. Primarily, cloud enables end users to service their own IT needs in a frictionless manner, making the business much more agile - as business units and individuals can innovate quickly and execute on new ideas immediately before they become stale. A secondary but still crucial benefit is that the cost of IT and its value become intimately tied together with a high degree of transparency. Costs are minimized and associated with end user organizations and business units. Let’s dive into the cost analysis in a bit more detail.

Capex is completely eliminated for any work that can be adequately served by a public cloud. For work that needs to be done inside an organization in a private cloud, capex expenditures become just-in time and are always justified by current and projected usage. The capex, plus operating expenses, is totaled and a per unit of time per resource cost is calculated. That cost is then shown back or actually billed to the different business units. This enables the business units to make sound and informed decisions on what to deploy and not to deploy based on the value of the work they are doing. This sort of calculus is always best decided by the business unit, not IT, as they are responsible for their own P&L. In this way, capex is reduced because the business unit is incentivized to only use what they need to drive the most value to the organization.

Opex costs like power, cooling and real estate are reduced as a function of consolidating the datacenters, pooling servers, reducing wasted capacity and incentivizing end users to make lower cost requests as just described above, but they will still remain a substantial cost.

However, the administrative expense component of datacenter opex can be brought much closer to zero through infrastructure automation. The provisioning of workloads and other opex that formerly belonged to central IT is moved to the business units, not as an IT operation, but as a part of the normal flow of the day to day activities they do to get their jobs done. This relieves the business from much of its opex and/or allows the business to reallocate IT staff to more value generating activity.

Besides value to the business as a whole, cloud provides another set of benefits to the IT team specifically which should motivate IT leaders to drive the cloud discussion inside their organizations. Simply stated, cloud can make IT loved again. Historically, IT brought in technology innovations that improved the lives of their internal customers – PC’s, networks, databases, client/server applications, e-mail, mobile computing, virtualization, etc. Now, services like Amazon’s AWS have set a new expectation of IT systems responsiveness. Fairly, or unfairly, end users are coming to expect instantaneous gratification of their IT desires without the need for planning, budgeting or security audits. As a result, they are becoming impatient with IT. They wonder why IT costs so much and takes so long to deliver. By implementing a good private cloud, IT can deliver a finite set of resources in an on-demand manner without compromising security or compliance. Hybrid clouds allow IT to extend their services to handle bursts of unplanned activity that the private cloud does not have the capacity to meet. By enabling offload to public cloud while simultaneously adopting policies and controls of who can do what in public clouds, IT can introduce public cloud as another tool in the IT toolkit without abdicating their traditional responsibility for the safety of data and applications. This will prevent the skunkworks use of public clouds that too many companies see when lines of business tries to circumvent IT. Clouds can make IT the hero again.

Getting Started With Your Cloud

Now that we know what cloud is and what we should expect from it here is a proposed journey to have the easiest onramp and highest value result.

Segregate Applications

First, you need to find the right applications for cloud. As we will describe in the next article in this series, the best fits are:

  • Scale-out load balanced applications with stateless instances
  • Batch processing applications
  • Test and dev for the above two and for more traditional IT applications

As for legacy stateful IT applications deployed in production, non-cloud solutions will suffice. In many cases server virtualization will help – it can lower capex and increase service levels. Well known solutions from VMware, Microsoft, Citrix, RedHat and others can help here.

Make sure you know which end customers and which workloads are the best early cloud candidates according to the ease with which they can move to cloud and the extent to which cloud provides them with real value.

Incrementally Add Projects to the Cloud

Don’t try to boil the ocean – pick the ideal project to start with and add more challenging projects as you experience success. Plan subsequent projects incorporating knowledge gained from previous projects.

Pick a Project Based on End-User Needs and Application Type

Within the applicable projects, pick one that has a burning pain, eager customers, small enough scale to experiment, but large enough scale to be a meaningful test and that aims for results that can be measured – in terms of cost, speed to get results, lowered administration time, etc.

Pick a Private/Public/Hybrid Strategy for This Project

In a later article we will emphasize the need for your cloud software to support public, private and hybrid clouds. Assuming you have chosen a software partner that accommodates this choice, you need to make a decision for this particular project.

Choose public cloud if your project:

  • Has highly variable needs without other projects that are able to statistically offset it.
  • Is not expected to persist for an extended period of time and does not justify its own dedicated infrastructure.
  • Does not have significant needs for privacy, security, or regulatory compliance, unless there is a public cloud provider that specializes in delivering the right assurances for a project of this specific nature.

Choose private cloud if:

  • A single project or a sum of a few projects is expected to have flat or steadily growing compute needs.  Avoid projects that spike heavily and drop sharply unless they can be statistically offset by other projects that peak at different times.
  • Is it either a long-lived project or a short-lived project that, upon completion, will be replaced by projects of equal or greater resource needs.
  • Has significant needs for privacy, security, or regulatory compliance that can not be met by general-purpose or even specialty public cloud providers.

Choose hybrid cloud if the project can be meaningfully segmented into parts that have the private characteristics and other parts that have the public characteristics. Placing the right work in the right place will give you the best balance of cost, security, flexibility and return on investment.

Deploy Project

Deploying the project includes the following steps:

  • Engaging with the end user customer base to explain and sell the project.
  • Collaborate on end goals, desired metrics, and a timeframe for evaluation.
  • Acquiring, deploying and configuring physical infrastructure (if you chose private or hybrid cloud).
  • Choosing the cloud management software and deploying it.
  • Training the end-users and their management team on the self-service workflows – both for the delegation or rights and for the actual execution of work.


At regular points in the project, measure the results to the end user customer base and review with the customers. The goals need to be quantifiable and should be set at the beginning of the project. Goals may be around project completion time, time it takes to do specific tasks, overall cost or utilization of the infrastructure, etc. If the results are not what were expected, adjust the deployment to try to meet the goals. This can involve reconfiguring HW or SW, redoubling on customer training or adding in custom automation developed internally or from contractors.

At this point, it is also good to engage with the HW and SW providers to make sure they can identify any errors in deployment or deviation from best practices that are hampering the results. At this time, you will also have a better idea of your needs and will be in a position to make stronger and more prioritized feature requests.

Bring in Least Needful Applications for Sake of Conformity of Process, Service Levels, etc.

Only after you have had success in the ideal cloud use cases should you bring in the less applicable workloads. Though it may be harder to do so and the benefits may be less, over the long haul your IT department will benefit from a common infrastructure layer for datacenter management and a common end-user interface for deploying workloads, tracking their status and measuring their cost.

We're Hiring!Nimbula is looking for talented and self starter individuals to join and help with our mission to deliver EC2-like services behind the firewall.

Site Developed & Maintained by Unomena