Under the Hood of J2EE Clustering

Sky-Tiger

Memory-based session persistence stores session information in the memory of one or more backup servers instead of Database (shown as figure 8). This approach is very popular due to its high performance. Comparing with database approach, direct network communication between the original server and backup servers is really lightweight. And also note that in this approach, the “restore” phase in Database persistence approach is not needed because after session backup, all session data is already in the backup servers’ memory for the coming requests.

”JavaGroups” is currently the communication layer of JBoss and Tomcat clustering. JavaGroups is a toolkit for reliable group communication and management. It provides core features such as “ Group membership protocols” and “message multicast”, which is very useful in making clustering work. For more information about “JavaGroups”, please refer to “http://www.jgroups.org/javagroupsnew/docs/index.html”.

Tomcat’s approach :Multi-servers replication
Many variations of memory replication exist. The first method is replicating the session data across all of the nodes in the cluster. Tomcat 5 implements memory replication in this way.

Sky-Tiger

Shown as figure 9, when sessions change in one server instance, it will backup its data to all other servers. When one server instance fails, the load balancer can choose any of other available server instances as its backup. But this approach has some limitations in scalability. If there are too many instances in a cluster, the network communication cost cannot be ignored, it will decrease the performance heavily and network traffic may also be a bottleneck problem.

Weblogic, Jboss and WebSphere's approach-- paired servers replication
For performance and scalability reasons, Weblogic, JBoss and Websphere all provide another way to perform memory replication: each server instance chooses an arbitrary backup instance to store session information in-memory, shown as figure 10.

In this way, every server instance has its own paired backup server instead of all other servers. This approach eliminates the scalability problems when more instances are added to the cluster.

Sky-Tiger

Although this approach brings an implementation of session failover with high performance and high scalability, it still has following limitations:

It brings more complexity to load balancer. When one server instance fails, the load balancer must remember which instance is the paired backup server of the failed one. This will diminish the scope of load balancers, and some hardware boxes cannot be used in such requirement.
In addition to normal request processing, the servers are taking on replication responsibility as well. Per Server instance, request processing capacity is diminished because some of the CPU cycles are now going toward replication duties.
In normal processing, a lot of memory which is used to store backup sessions wastes in every backup servers when no session failover happens. This can also increase JVM’s GC overhead.
The server instances in a cluster form replication pairs. So if the primary server which the sessions is stuck to fails, the load balancer can send all failover requests to the backup server. The backup server will see doubling in incoming requests after the primary fails and this will cause performance problems in the backup server instance.
To overcome above limitations, variations from different vendors come into being. To overcome the 4 th point above, Weblogic defines the replication pairs for each session instead of each server. When one server instance fails, the sessions in the failed server have dispersed backup servers, and load gets evenly spread after failure.

IBM’s Approach -- centralized state server
Websphere has an alternative choice to memory replication: Backup Session information to a centralized state server, shown as figure 11.

Sky-Tiger

It’s very similar to the database solution. The difference is that a dedicated “Session backup Server” replaces the database server. This approach brings combined advantages from both database and memory-replication solutions:

Separating requests processing from session backup processing makes the cluster more robust.
All the session data is backed up to the dedicated servers. No need for server instances to waste memory for backing up session data from other servers.
Session can fail over to any other instances, since the session backup servers are shared by all nodes in the cluster. So, most of load balancer software and hardware can be the choice for the cluster; and more important, the request loads will spread evenly when one server instance fails.
Because the socket communication between Application server and session backup server is lightweight comparing to the heavy database connections, it has better performance and is more scalable than the Database solution.
However, due to the “restore” phase to recover the session data for the failed server, its performance cannot be the same as the solution in which memory is replicated directly between pairs. Also, the additional session backup servers add more complexity to administrators, and it is more likely to form the performance bottleneck in the backup sever itself.

Sun’s approach – special Database

Sky-Tiger

Sun JES Application Server implements Session Failover differently,shown as figure 12. From the surface, it looks like the same as the database approach, because it uses a relational database as the Session store and use JDBC to access all the session data. But from the internal, the relational database used by JES, which is called HADB, is optimized for session access specially and stores almost all data in memory. So, you can say it is more close to the approach of centralized state server.

Figure 12: Special Database Replication

Performance issues
Think about this: One Web server may host dozens of Web applications, each of which may be accessed by hundreds of concurrent users, and every user will generate his browser sessions to access certain applications. All the session information needs to be backed up in case of the server failure to restore the sessions in another available server instance. And even worse, the sessions are changing from time to time: when sessions are created or expired, when attributes are added to or remove from the sessions and when attributes are modified; even when no attributes are updated, the last modified time of the session is changing when accessing (to decide when to expire the session). So the performance is the big issue in the session failover solutions. Vendors always give you some tunable parameters to adjust the server’s behavior to meet your performance requirement.

When to backup sessions

When client requests are processed, the session data is changing every time. For performance issue, it is not wise to backup sessions in real time. It is really a trade-off to choose the backup frequency. If backup actions take place too frequently, the performance will be impacted sharply; But the longer the interval between two backup actions is, the more session information you will lose if a server failure happens during the interval. In spite of all approaches, including database and memory replication, followings are popular options to decide the backup frequency:

By Web-methods.
The session state is stored at the end of each Web request prior to sending a response back to the client. This mode provides the best guarantee that the session state is fully updated in case of failure.
By Constant interval.
The session state is stored in the background at the frequency of a constant interval. This mode does not guarantee that session state is fully updated. However, it can provide a significant performance improvement because the state is not stored after each request.

Sky-Tiger

Backup granularity

When backing up sessions, you still have choices to decide how much of the session state is stored. Some common choices among different products are:

Whole session.
The entire session state is stored every time. This mode provides the best guarantee that your session data is correctly stored for any distributable Web application. This approach is simple and adopted in both memory replication solution and Database persistence approach by default.
Modified session.
The entire session state is stored if it has been modified. A session is considered to have been modified if “HTTPSession.setAttribute()” or “HTTPSession.removeAttribute()” was called. You must guarantee that these methods are called every time an attribute is changed. This is not a J2EE specification requirement, but it is required for this mode to work properly. Modified session backup cuts down the number of sessions to be stored. Those requests which only read attributes from sessions will not trigger the session backup actions, that brings better performance than the whole session mode.
Modified attribute.
Only modified session attributes are stored instead of the whole session. This minimizes the session data to be backed up. This approach will bring the best performance and least network traffic. For this mode to work properly, you must follow a few guidelines. First, Call “setAttribute ()” every time the session state is modified and only the modified object is serialized and backed up. Second, Make sure there are no cross-references between attributes. The object graph under each distinct attribute key is serialized and stored separately. If there is any object cross references between the objects under each separate key, they will not be serialized and deserialized correctly. For example, in a memory replication cluster, shown as figure 13, there are a “school” object and a “student” object in the session and the “school” object has a reference to the “student”. The “school” object is modified at sometime and backup itself to the backup server. After serialization and deserialization, the restore version of “school” object in the backup server will include the whole object graph and contain a “student” object with a reference to it. But the “student” object can be modified separately. When it is modified, only the “student” itself will be backed up. After serialization and deserialization, a “student” object is restored in the backup server’s memory, but at this time, it will lose the connection with “school” object. Although this approach brings the best performance, the above limitation is imposed on Web application's architecture and design, especially if you are using the session to store cached complex user data.

Sky-Tiger

Other failover implementations
As I mentioned in the above section, granularity is very important to performance when sessions are backed up. However, current implementations, both database persistence and memory replication, are all using Java object serialization technology to transfer the java objects. This will bring a big footprint, impact system’s performance and also give a lot of limitations on Web application's architecture and design. Some J2EE vendors seek special means to implement web clustering in a lightweight, small footprint mode and provide fine-granularity distributed-object sharing mechanism to improve cluster performance.

JRun with Jini

JRun 4 has built their clustering solution based on Jini technology. Jini was born for distributed computing and it allows you to create a "federation" of devices and software components in a single distributed computing space. Jini provides the distributed system services for look-up, registration, and leasing which is useful to a clustering environment. Another technology called JavaSpace built on Jini provides features such as object processing, sharing, and migration which are also valuable to a cluster implementation. For more information about Jini and JavaSpace, please refer to “http://java.sun.com/products/jini/2_0index.html”.

Tangosol with Distributed Cache

Tangosol Coherence™ provides a distributed data management platform which can be embedded into most of popular J2EE containers to provide clustering environment. Tangosol Coherence™ also provides distributed cache system which can share java objects among different JVM instances effectively. For more information about Tangosol, please refer to “http://www.tangosol.com/”.

Sky-Tiger

JNDI clustering implementation
The J2EE specification requires that all J2EE containers should provide an implementation of the JNDI specification . The primary role of JNDI in a J2EE application is to provide the indirection layer so that resources can be found without being much aware of the indirection. This will make J2EE components more reusable.

Having full-featured clustered JNDI is important for a J2EE cluster, as almost any EJB access starts with a lookup of its home interface in the JNDI tree. Vendors implement JNDI clustering differently, depend on their cluster structure.

Shared global JNDI Tree
Both Weblogic and JBoss have a global, shared, cluster-wide JNDI Context that clients can use to lookup and bind objects. Things bound to the global JNDI Context will also be replicated across the cluster through IP multicast so that if a server instance is down, the bound objects will still be available for lookup.

Sky-Tiger

Shown as figure 14, the shared global JNDI tree actually consists of all the local JNDI deposits in every node. Each node in a cluster hosts its own JNDI naming server, which replicate everything to the other naming servers in the cluster. Thus, every naming server has a copy of every other naming server’s objects in the tree. This redundant structure makes the Global JNDI Tree highly available.

In practice, this clustered JNDI tree can be used for two purposes. You can use it for deployment which is the task of administrator. After deploying EJB modules or setting JDBC&JMS services in one server instance, all the objects in the JNDI tree will be replicated to other instances. During runtime of applications, your programs can access JNDI tree to store and retrieve objects by using JNDI API, and your custom objects are also be replicated globally.

Independent JNDI
While JBoss and Weblogic all adopt global shared JNDI, Sun JES, IBM Websphere and others utilize an independent JNDI tree for each application server. Member servers in an independent JNDI tree cluster do not know or care about the existence of other servers in the cluster. Does this mean they don’t want clustered JNDI? As almost any EJB access starts with a lookup of its home interface in a JNDI tree, the clustering features would be almost useless without a clustered JNDI tree.

Actually, the independent JDNI tree can still have highly available features only if their J2EE application is homogeneous. We call it a homogeneous cluster when all the instances in the cluster have the same settings and have deployed the same set of applications. Under such condition, special admin tools called agent can help achieve the high availability, shown as figure 15.

Sky-Tiger

Both Sun JES and IBM Websphere have node agent installed on each instance in the cluster. When deploy EJB modules and binding other JNDI services, the admin console can send commands to all agents to achieve the same effect of the global shared JNDI tree.

But the independent JNDI solution will not support replication for arbitrary objects which are bound and retrieved by running applications. They have reasons for this: the primary role of JNDI in a J2EE application is to provide the indirection layer for administrating external resources, not for runtime data deposits. If such requirements happen, an individual LDAP server or database with HA features can help. Both Sun and IBM have their own individual LDAP server products which are already shipped with clustering features.

Centralized JNDI
A few of J2EE products use centralized JNDI tree solution in which the naming server is hosted on a single server and all servers instances register their same EJB components and other admin objects on the single naming server.

The naming server itself implements highly available features which is transparent to client. All clients look up EJB components in this single naming server. But this structure always implies complex installation and administration and has thus been abandoned by most vendors.

Initial access to JNDI server
When clients to access the JNDI server, they need to know the hostname/IP address and port number of the remote JNDI server. In global and independent JNDI tree solutions, there are more than one JNDI servers. Which one should clients connect to for the first access to JNDI servers? How to achieve load balancing and failover?

Technically, a software or hardware load balancer can sit between the remote clients and all JNDI servers to perform load balancing and failover. But few vendors implement this way, there are many simple solutions.

Both Sun JES and JBoss implement JNDI clustering by making the “java.naming.provider.url” JNDI setting accept a list of URLs separated by a comma. For example, java.naming.provider.url=server1:1100,server2:1100,server3:1100,server4:1100
The client will try to get in touch with each server from the list, one after the other, stopping as soon as one server has been reached.
JBoss also has implemented auto-discovery features. When property string “java.naming.provider.url” is empty, the client will try to discover a bootstrap JNDI server through a multicast call on the network.