Building cloud-ready, multicore-friendly applications

Sky-Tiger

Another difficulty arises from managing things at the granularity of a virtual machine image. In typical infrastructure-as-a-service cloud environments, you create a virtual machine image and populate that image with services, applications, and configurations. If something changes in your application's configuration, you revise the virtual machine image, a time-consuming and potentially error-laden process. The term image sprawl aptly describes the growing pool of images that result from this model.

Some cloud application platforms provide a finer degree of granularity. With such platforms, you address the virtual machine image as what it should be: an operating system layer plus a cloud application platform agent. The agent oversees the virtual machine, and allows services, applications, and configurations to be deployed, updated, and versioned, and their lifecycles maintained, without changing the underlying virtual machine image. The difference between these two image-management styles is illustrated in Figure 6.

Coarse-grained vs. fine-grained image management
Figure 6. Coarse-grained vs. fine-grained image management. (Click to enlarge.)

Why do I even bring this up in an article primarily focused on developers? Deployment is not a production- and testing-only concern. Anything that affects time usage during development needs to get the hairy eyeball -- at least, my pragmatic-programmer roots think so.

Of course, this level of manageability goes way beyond the developer. I'm aware of one company with over three hundred workers (and over five hundred cores) that manages its private production cloud running multiple cloud applications with less than one-third of an administrator's time.

Sky-Tiger

Load balancing

Cloud platforms use various types of load balancing. It may be as simple as using software- or hardware-based load balancers between the cloud application and its client -- or it could be as sophisticated as the cloud platform utilizing its own built-in software-based load balancing. Load balancing affects both scalability and availability. When your application's work is distributed across many workers, you want to make sure that the resources of each worker in the cloud are fully utilized when needed. It would not help you if some workers were maxed out and others ignored or underutilized while your application is under a spike of heavy load.

The situation gets more complex when you introduce servers with varying capacities and speeds into your cloud. If your cloud is made up of workers that range from older single-core processors up to machines with four or more cores, then you have workers with very different capability footprints. As demand ramps up, then the CPU cores, memory, and other resources should be utilized fairly across the distributed workers. Slower machines should carry the load they are capable of, and faster machines should be utilized more. The old naval adage that the armada is as fast as its slowest ship should not apply here.

Either way, wouldn't it be great if you as a developer did not have to worry about this? Some cloud platforms take care of this for you, so you don't have to sweat the application infrastructure or architecture to make sure the application load balances across the cloud.

Sky-Tiger

Command and control

Using atomic, cohesive code opens up the possibility of using declarative state machines. Declarative state machines have been around for a while; they allow you to design flows of steps in a declarative way, often in XML or some other domain-specific language (DSLs). They are often used in middleware and in defining business logic and workflows. Spring Web Flow is based on this concept, as is Microsoft Windows Workflow, and Appistry's own Process Flow technology. There are many other examples.

Typically, the model runs something like this: a state machine of different steps is defined. Each step or state is tied to a task or unit of executable logic. The state to which the machine branches is determined by the task execution results of the prior step. If a step succeeds, then the next step takes some happy path. If a step fails, then the next step may execute a compensating task to deal with the failure, or request help, or return failure. Success or failure is usually defined by conditional logic, rules, data values, thrown exceptions, and other conditions.

By using declarative state machines to orchestrate atomic, stateless, and perhaps idempotent code in the context of distributed environments like cloud platforms, you can get surprising levels of robustness, reliability, and flexibility. Additionally, if your code must be stateful or cannot be safely re-executed because its operations are not idempotent, the declarative state machine makes that code more reliable. The declarative nature of the state machine allows your design to accommodate failures in these conditions, without putting the failure handling inside your code. Also, some state machines allow for snapshooting progress in the state machine steps, so that a process interrupted by failure can be resumed and completed. Again, this is something that would not be possible without making sure the code breaks down nicely into atomic steps or tasks.

When this technology is seen in cloud platforms, it allows the orchestration of your code across many workers in a reliable, scalable, available, and load-balanced way without your code knowing about it.

Sky-Tiger

Where to from here

Do cloud platforms exist in the real world? There are differences from platform to platform, both in features and focus, but typically each cloud platform hides the cloud infrastructure from your application, virtualizes your application to manage and leverage it in a cloud-like manner, and provides essential services so that you as a developer do not need to re-invent the wheel.

You should now have a good sense of how cloud computing will change the way you design code. Like any design principles, the ones described in this series have to be applied with some common sense. There are no magic or silver bullets, and hammers aren't the right tool for every job. However, considering these principles will help you leverage your code on cloud platforms now and in the onrushing future.

See the Resources section below to learn more about cloud computing.

Sky-Tiger

Multicore processing power and cloud computing are two of the most exciting challenges facing software developers today. Multiple chips or processing cores will enable individual computing platforms to process threads unbelievably fast, and the advent of cloud computing means that your applications could run on multiple distributed systems. In this first half of a two-part article, Appistry engineer Guerry Semones gets you started with the four design principles for writing cloud-ready, multicore friendly code: atomicity, statelessness, idempotence, and parallelism. Level: Intermediate

With the advent of cloud computing, the explosive growth of mobile device use, and the growing availability of multicore CPUs, there is a drive toward new models for designing and developing code. Well, they're not new, really. Some of the models have been around, and in use, for some time, but they're not typically practiced by mainstream developers ... yet.

Michael Groner wrote about this trend in a blog post entitled "Microsoft says you need to change how you are building your applications." Here is part of what he said:

    I was surprised how many speakers [at Microsoft TechEd 2008] were conveying the same message: CPU speeds are topping out. If you want your applications to run faster and better you are going to have to build your applications in a new way. The solution isn't just to learn how to multi-thread your applications. The solution lies in building your applications into smaller units of code called tasks that can be moved around to the different cores of a multicore machine.

The message that the time had come to "take programs and break them down into parallel execution units" was music to Michael's ears.

    Why? It is the same model that we [at Appistry] have been using for fabric computing since our beginnings in 2001. Now we are starting to see this message more and more as cloud computing is gaining acceptance.

For your applications to meet the demands of computing in a cloud-based, multicore world, you'll need to design your code with the following attributes in mind:

    * Atomicity
    * Statelessness
    * Idempotence
    * Parallelism

In this article, the first of two parts, I'll discuss each attribute in turn and then explain what it means in practice, when you write code. In the next article you'll get a glimpse of how applications written to maximize these four attributes can reap the benefits of various multi-processor, distributed architectures.

Sky-Tiger

Atomicity

An atomic piece of code has a specific and clearly defined purpose. In object-oriented terminology, it has cohesion. Atomic code adheres to Robert Martin's single responsibility principle, where not only classes but methods have distinct jobs, each method having a single concern. Such code is like a delicious, pepperoni pizza, not the garbage pizza loaded with everything. Additionally, atomic code does not rely on a specific order of execution.

Sky-Tiger

Don't lean on me

While atomic code may require other libraries, its execution is self-contained -- that is, it contains no call-level interdependencies. If there is interdependency across individual method calls, then your individual methods are not atomic, though they may form an atomic operation all together. Consider the simple example in Listing 1. A call on the setRadius() method must precede a call on any of the computational methods, such as getDiameter(). Therefore, setRadius() and getDiameter() are not atomic separately. Though this is fine from a purely object-oriented design standpoint, it has implications in distributed and parallel environments. The computational methods in Listing 1 cannot be processed in parallel across distributed workers or cores independent of setRadius().
Listing 1. A simple class with call-level interdependencies

package com.appistry.samples;

public class NonAtomicCircle {

    private int radius;

    public void setRadius(int radius) {
        this.radius = radius;
    }

    // setRadius must be called before any of the following

    public float getArea() {
        return 3.14f * (radius * radius);
    }

    public int getDiameter() {
        return 2 * radius;
    }

    public float getCircumference() {
        return 3.14f * getDiameter();
    }
}

The code in Listing 2, though more static in nature, does not have this dependency or its associated restrictions, and can be executed independently. In more complex classes involving equally complex state, this can become a critical design consideration.
Listing 2. A simple class with atomic methods

package com.appistry.samples;

public class AtomicCircle {

    private int radius;

    public void setRadius(int radius) {
        this.radius = radius;
    }

    // These computational methods can called independently
    // and distributed or run in parallel.

    public static float getArea(int radius) {
        return 3.14f * (radius * radius);
    }

    public static int getDiameter(int radius) {
        return 2 * radius;
    }

    public static float getCircumference(int radius) {
        return 3.14f * getDiameter(radius);
    }
}

Sky-Tiger

It's simple, my dear Watson

Atomic code is also concise by its nature, and, as stated above, has a specific, clearly defined purpose. Fat, hairy, do-multiple-things methods like provisionLineItem() in Listing 3 are likely not atomic (though the methods it delegates to are candidates). If a method serves multiple purposes, then you should break those purposes up into separate atomic methods, and likely into separate classes, as reserveInventory() and calculateWeight() sound like separate concerns.
Listing 3. A non-atomic method with too many responsibilities

package com.appistry.samples;

public class OverAchiever {

    public void provisionLineItem(Shipment shipment, String sku, int quantity) {
        Product item = reserveInventory(sku, quantity);
        float weight = calculateWeight(item, quantity);
        shipment.add(item, quantity, weight);
        reportPopularityToCorporate(sku, quantity);
    }
   
}

The same holds true for long-running methods: they are not usually atomic. Long-running code may have a single purpose, but if you can break the method down into more atomic steps, then you gain flexibility when running that code in cloud environments or on multiple cores.