It should be noted that this is not an endorsement for Hazelcast.  Hazelcast is a very good open-source product in this space, but there are other products, both commercial and open-source, that offer the same features or more.  We chose Hazelcast for this post because its a lightweight, open-source solution with an easy to understand API.

Two best of breed commercial products in this space are VMware vFabric Gem Fire and Oracle Coherence.  Give both of these products a look (or let me know and I’ll help you) if you’re interested in enterprise-grade in-memory data grids.  Our example uses Hazelcast, but the concepts are applicable to other in-memory data grids.

Prerequisites

There are a few assumptions and prerequisites for this example.  For this example, we’re going to create a Java main class that starts the Hazelcast datagrid.  The project is created in Spring Source Tool Suite, uses Maven 2 for build and dependency management, and uses Spring 3.x for dependency injection, and MySQL for persistence.  If you’re not comfortable with any of these technologies, it’s probably worthwhile to become familiar with them.

Maven Configuration

The first step is to add the dependencies for Hazelcast, Spring Framework, and MySQL to your Maven POM.  Of course, adjust the versions and Spring and MySQL dependencies accordingly:

<dependencies>
  <dependency>
    <groupId>com.hazelcast</groupId>
    <artifactId>hazelcast</artifactId>
    <version>1.9.3.3</version>
  </dependency>
  <dependency>
    <groupId>org.springframework</groupId>
    <artifactId>spring-core</artifactId>
    <version>3.0.5.RELEASE</version>
  </dependency>
  <dependency>
    <groupId>org.springframework</groupId>
    <artifactId>spring-jdbc</artifactId>
    <version>3.0.5.RELEASE</version>
  </dependency>
  <dependency>
    <groupId>commons-dbcp</groupId>
    <artifactId>commons-dbcp</artifactId>
    <version>1.4</version>
  </dependency>
  <dependency>
    <groupId>mysql</groupId>
    <artifactId>mysql-connector-java</artifactId>
    <version>5.1.16</version>
  </dependency>
</dependencies>

Domain and DAO Classes

The next step is to create the domain and data access classes for accessing our database and back the Hazelcast cache.  For this example, we’re going to use a music artist whose information is already stored in the database.  The domain model is simple, but can be easily extended.  We’re also using Spring’s JDBC template for data access; any data access technology can be used, including JPA and JDO.  There’s nothing very interesting about these classes.

The domain class is:

package net.avantia.datagrid.domain;

/**
 * Domain object for artist.
 * @author Brian Jimerson
 *
 */
public class Artist {

	private Integer id;
	private String name;
	private String url;

	/**
	 * @return the id
	 */
	public Integer getId() {
		return id;
	}
	/**
	 * @param id the id to set
	 */
	public void setId(Integer id) {
		this.id = id;
	}
	/**
	 * @return the name
	 */
	public String getName() {
		return name;
	}
	/**
	 * @param name the name to set
	 */
	public void setName(String name) {
		this.name = name;
	}
	/**
	 * @return the url
	 */
	public String getUrl() {
		return url;
	}
	/**
	 * @param url the url to set
	 */
	public void setUrl(String url) {
		this.url = url;
	}
}

And the DAO is:

package net.avantia.datagrid.dao;

import java.sql.ResultSet;
import java.sql.SQLException;
import java.util.List;

import javax.sql.DataSource;

import net.avantia.datagrid.domain.Artist;

import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.jdbc.core.RowMapper;

/**
 * JDBC Template DAO for artists.
 *
 * @author Brian Jimerson
 *
 */
public class ArtistJdbcDao {

	private JdbcTemplate jdbcTemplate;

	/**
	 * Constructs a new ArtistJdbcDao with the specified data source.
	 * @param dataSource The data source to use.
	 */
	public ArtistJdbcDao(DataSource dataSource) {
		this.jdbcTemplate = new JdbcTemplate(dataSource);
	}

	/**
	 * Gets all artists.
	 * @return All artists
	 */
	public List<Artist> getAllArtists() {
		List<Artist> allArtists = this.jdbcTemplate.query(
				"select * from artist", new ArtistMapper());
		return allArtists;
	}

	/**
	 * Finds an artist by id.
	 * @param id The id of the artist to find.
	 * @return The artist if found.
	 */
	public Artist findArtistById(Integer id) {
		Artist artist = this.jdbcTemplate.queryForObject(
				"select * from artist a where a.id = ?", new Object[]{id}, new ArtistMapper());
		return artist;
	}

	/**
	 * Artist mapper implementation of RowMapper
	 * @author Brian Jimerson
	 *
	 */
	final class ArtistMapper implements RowMapper<Artist> {

		/**
		 * Maps a result set row to an Artist object.
		 */
		public Artist mapRow(ResultSet rs, int rowNum) throws SQLException {
			Artist artist = new Artist();
			artist.setId(rs.getInt("id"));
			artist.setUrl(rs.getString("url"));
			return artist;
		}
	}
}

Map Loader

Our implementation of MapLoader is what tells Hazelcast how to back it’s Map cache entries with our database entries.  Our example only needs read access to data (not write access to the database) so we only have to implement a MapLoader.  The code for our MapLoader is:

package net.avantia.datagrid.util;

import java.util.Collection;
import java.util.HashMap;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Set;

import org.springframework.context.support.ClassPathXmlApplicationContext;

import net.avantia.datagrid.dao.ArtistJdbcDao;
import net.avantia.datagrid.domain.Artist;

import com.hazelcast.core.MapLoader;

/**
 * MapLoader implementation for load artists into Hazelcast.
 *
 * @author Brian Jimerson
 *
 */
public class ArtistMapLoader implements MapLoader<String, String> {

	private ArtistJdbcDao dao;

	/**
	 * Default constructor for ArtistMapLoader
	 */
	public ArtistMapLoader() {
		ClassPathXmlApplicationContext ctx = new ClassPathXmlApplicationContext("avantia-data-grid-context.xml");
		dao = ctx.getBean("artistJdbcDao", ArtistJdbcDao.class);
	}

	public String load(String id) {
		Integer intId = Integer.parseInt(id);
		Artist a = dao.findArtistById(intId);
		if (a != null) {
			return a.getName();
		} else {
			StringBuilder sb = new StringBuilder();
			sb.append("Artist for id [");
			sb.append(id);
			sb.append(" not found.");
			return sb.toString();
		}
	}

	public Map<String, String> loadAll(Collection<String> ids) {
		Map<String, String> artistMap = new HashMap<String, String>();
		List<Artist> artists = dao.getAllArtists();
		for (Artist a : artists) {
			artistMap.put(a.getId().toString(), a.getName());
		}
		return artistMap;
	}

	public Set<String> loadAllKeys() {
		Set<String> allArtistIds = new HashSet<String>();
		List<Artist> artists = dao.getAllArtists();
		for (Artist a : artists) {
			allArtistIds.add(a.getId().toString());
		}
		return allArtistIds;
	}

}

This MapLoader returns a Map to be cached; the key of the Map entry is the artist id, and the value is the artist name.  You could easily make the the value of the entry be an Artist bean with all of the attributes of the artist by changing the implementation and method signatures.

Configuration

The next step (almost done!) is to configure Hazelcast and our Spring beans.  The only thing special for our Hazelcast configuration is the class name for our map called 'artists'.  Everything else is the standard Hazelcast configuration:

<!-- Regular Hazelcast configuration omitted -->

<map name="artists">
  <map-store enabled="true">
   <class-name>net.avantia.datagrid.util.ArtistMapLoader</class-name>
   <write-delay-seconds>0</write-delay-seconds>
  </map-store>
</map>

And our Spring configuration is standard too; it just configures our data source and DAO:

<bean id="dataSource" destroy-method="close">
 <property name="url" value="jdbc:mysql://localhost:3306/artists" />
 <property name="username" value="root" />
 <property name="password" value="password" />
 <property name="driverClassName" value="org.hibernate.dialect.MySQLDialect" />
</bean>

<bean id="artistJdbcDao">
 <constructor-arg ref="dataSource"/>
</bean>

<context:component-scan base-package="net.avantia" />

Main Class

The final class for the data grid server is the Java main to start Hazelcast. This could also be a Servlet in an application server or other ways of starting a listener.

Our Main class is:

package net.avantia.datagrid;

import com.hazelcast.config.ClasspathXmlConfig;
import com.hazelcast.config.Config;
import com.hazelcast.core.Hazelcast;
import com.hazelcast.core.HazelcastInstance;

/**
 * Starts the cache server.
 * @author Brian Jimerson
 *
 */
public class DataGridServer {

	private HazelcastInstance instance;

	/**
	 * Main entry point for the memory data grid server.
	 * @param args
	 */
	public static void main(String[] args) {
		Config cfg = new ClasspathXmlConfig("hazelcast.xml");
		HazelcastInstance instance = Hazelcast.newHazelcastInstance(cfg);
		DataGridServer server = new DataGridServer(instance);
		int status = server.run();
	}

	/**
	 * Creates a new CacheServer instance.
	 * @param instance The underlying HazelcastInstance
	 */
	private DataGridServer(HazelcastInstance instance) {
		this.instance = instance;
	}

	/**
	 * Runs the server.
	 *
	 * @return 0 if run completes successfully; a negative number if there was an error.
	 */
	private int run() {
		System.out.println("Started data grid server with instance [" + instance.getName() + "].");
		return 0;
	}

}

Data Grid Client

Once you start the data grid server, there are a number of ways to access the data grid. Hazelcast provides Java, memcached (yes, memcached client access, but it doesn’t seem to support named maps), and REST clients out of the box, and these should cover most of your needs.

For Java client access, you can either wire up a Spring bean like this:

	<bean id="hazelcastClient"
		class="com.hazelcast.client.HazelcastClient"
		factory-method="newHazelcastClient">
		<constructor-arg value="artist"/>
		<constructor-arg value="artist"/>
		<constructor-arg value="localhost"/>
	</bean>

or in Java like this:

  HazelcastClient hazelcastClient = HazelcastClient.newHazelcastClient("artist", "artist", "localhost");

And once you have a client instance you can get all of the grid’s artists like this:

  Map<String, String> allArtists = hazelcastClient.getMap("artist");
  System.out.println("All artists = " + allArtists);

Conclusion

In this post, we’ve walked through the process of using Hazelcast as an in-memory data grid, providing a map cache backed by a persistent store. This cache is read-only, but could easily support massive reads with little latency in a cloud-based infrastructure.

There are many other use cases for in-memory data grids in the cloud: writing to persistence-backed caches, name-value entries, and map-reduce strategies. This is just one use case that shows how to leverage a traditional database to support large-scale data access for persistent data.

Tier-Based distribution is similar to SOA, albeit superficially. The cornerstone of SOA is cohesive services with clearly defined boundaries. In an SOA system boundaries are drawn along vertical lines delineating areas of responsibility. N-tier design uses boundaries to define units of scalability rather than areas of responsibility. Boundaries between tiers are drawn along fault lines in the resource landscape – hand-off points between components. A business process that might fall under a single service in SOA might live in multiple tiers to allow optimization of resource usage at critical points in the process. Loosely coupled units of related functionality may scale independently in an n-tier system. This flexibility comes at the cost of artificial boundaries injected into an otherwise cohesive process with forced integration to allow the process to flow across those boundaries.

An example of such artificial boundaries is found in nearly every design pattern in common use. MVC allows each part to vary (and scale) independently by separating models, views, and controllers into independent tiers. Whether deployed separately or together these independent tiers operate in a mutually dependent integration as defined by a business process. Physical separation and logical independence do not break the links between steps in a process. Even a distributable process remains a cohesive unit when viewed as a process as opposed to a pattern.

Resource-Based distribution relies on process-oriented cohesion, creating a more natural separation of functionality along process boundaries. As with SOA, boundaries delineate areas of responsibility. Scalability is achieved not through artificial separation of functionality, but organically through process improvement. Separation of concerns is aligned with business needs rather than the organization imposed by programing constructs.

Organic growth tightly coupled to a business process is here assumed a desirable outcome. Tier- and resource-based distribution are conceptually similar in their ability to scale. Both may allocate computational resources to any subprocess as needed. Tier-based distribution errs toward flexibility and scalability in its interpretation of a business process. When the business process changes, the focus on scalability results in divergence with the process. Resource-based distribution errs toward adherence to the business process. Changes in the process do not result in a loss of scalability. Incongruence with the business means that the system does not fulfill its intended purpose and can result in loss of information. Shouldn’t a system focus on the business process and scale naturally as the business grows?

Most of these data storage solutions require persisting data to an underlying disk or other permanent store medium.  They differ in their data structures and query mechanisms, but still must eventually store the data somewhere semi-permanent, so that the data can persist across restarts, failures, and backup/restore processes.  The downside to this is well known; disk and network access is slow.  Cloud applications are usually designed to support massive scale, such as tens of thousands of users or millions of queries per day.  Reading and writing data from disk or network doesn’t lend itself to this kind of scale.

A different type of data solution that can support this kind of scale is a distributed in-memory data grid.  A distributed in-memory data grid stores data in memory, as the name says, which performs notoriously well.  Most traditional caching mechanisms use memory access to provide performance.  Even though traditional cache solutions provide fast in-memory data access, they are designed to perform a different function than in-memory data grids, and there’s a couple of things that they usually lack for cloud applications:

1. Since they were designed to do things like cache database query results or HTTP sessions, their data structures are usually limited to key/value pairs.
2. As traditional cache solutions were designed to be ‘embedded’ in applications, its hard to scale traditional cache solutions in a grid format.  Most traditional cache solutions support horizontal scaling through cluster communications and replication, but they don’t support vertical scaling of data segments through techniques like sharding and MapReduce.

So wouldn’t it be nice to use a data access solution that gave you:

• The performance of in-memory data access
• The reliability of data persistence
• The scalability of cloud-like grid clusters
• The flexibility of multiple data structures and programming models

That’s where distributed in-memory data grids come in.  These products give you the performance of memory access, like traditional cache systems.  However, they also give you the benefits of traditional database systems, like persistence and data structures, and they also add grid scale.  The idea is that with enough nodes in the data grid cluster, and the right replication scheme, storing data in-memory doesn’t mean the data is volatile.

With traditional data center architectures, in-memory data is never considered sufficient, because the restart of a server, failure of a server, bad hardware, or even an operating system update would cause a loss of data.  But if  you have a 100-node cluster, and if your data is sharded and replicated across the 100 nodes properly, you should be able to have access to all of the data even if you lose 5 or 10 nodes to any of these downtime scenarios.  What are the chances of enough nodes being down (say 50-60 out of 100) to lose any in-memory data?

If the probability of data loss can be reduced to a negligible factor, then not storing data in-memory becomes a hard argument.  In-memory data storage provides a lot of benefit to scale and performance, with very little risk.  Two things that are required are a large grid cluster (PaaS), like VMware Cloud Foundry or Amazon EC2, and an in-memory data grid solution designed for this application.

Hopefully I’ve convinced you that there is a strong case for looking at distributed in-memory data grids for large scale, public and private cloud applications.  Some of the in-memory data grids designed for this scenario are:

• VMware vFabric GemFire — GemFire is a best-of-breed solution for commercial applications.  Features like continuous queries are first class.
• Oracle Coherence — Coherence is another best-of-breed in-memory data grid.  If you’re using WebLogic Server 11g, you already have Coherence.
• Hazelcast — Hazelcast is an open-source, Java-based in-memory data grid.  It supports many of the features discussed, and some other standard features of in-memory data grids (like continuous queries) are planned.

In the next post, I’ll show you how to use Hazelcast to provide an in-memory data store, backed by a relational database, and how to query from a REST client.

Deployment Model Differences

Jahia is deployed as a standard WAR file, and, with the exception of classpath resources, template sets for sites are deployed as a directory in the WAR file’s templates folder. For example, if my template set is called ‘my-template-set’, the template set would be deployed to the Jahia WAR file’s templates/my-template-set directory. This deployment model is convenient, as it keeps the Jahia application, and custom templates, self-contained in a single deployable unit. When deploying templates to Jahia on Tomcat, this is as simple as copying template resources to the webapps/ROOT/templates directory in Tomcat. Tomcat treats folders in it’s webapps directory that match the exploded WAR format as contexts; the ROOT context (unless you’ve changed it) in Tomcat is the Jahia WAR file and is automatically served up as the / URL context.

WebLogic, like many full Java EE containers, do not have this kind of simple deployment model for a number of reasons. WebLogic assumes that there will be multiple servers, Admin or managed, in domains. It also supports clustered nodes and distributed deployments. Because of these features, deploying applications to WebLogic requires an application to be distributed through the Admin Server and node managers, and not just simply served up from the filesystem. What this means to Jahia administrators and developers is that updated template sets, which are the main artifact for Jahia development, can’t be copied to the Jahia WAR directory to deploy changes. When template sets are updated, the entire Jahia WAR file must be redeployed and redistributed through WebLogic Admin Servers and node managers, and as such eliminates the ability to do ‘hot deployment’ of template resources.

Jahia Config Wizard

The first step in deploying Jahia to WebLogic is to run the Jahia Config Wizard while its deployed to a temporary Tomcat server:

1. Download Jahia EE 6.1 tar file.
2. Extract Jahia to a temporary location.
3. Create a new user/schema for Jahia in Oracle DB.
4. Start Jahia by running <jahia-dir>/bin/startup.sh|.bat.
5. Open a browser and go to http://localhost:8080/config
6. Go through the Config Wizard; the key here is to set up the Oracle database properly instead of the default Hypersonic database.
7. Shutdown Jahia: <jahia-dir>/bin/shutdown.sh|.bat

Creating the Jahia EAR Structure

Since WebLogic is a full Java EE container, it expects EAR resources instead of WAR files. You can deploy WAR files directly to WebLogic, but behind the scenes is creates an EAR file for the WAR, and there are some application-level configuration settings necessary for Jahia on WebLogic, so you need to create an exploded EAR structure for Jahia:

1. Create a new WebLogic domain with an Admin Server and a managed server. We’ll call our domain jahia_domain and our managed server jahia_server1.
2. Copy the following files from the temporary Tomcat/Jahia server to the domain’s lib directory (<middleware-home>/user_projects/domains/jahia_domain/lib):
   <tomcat-dir>/lib/activation-1.1.jar
<tomcat-dir>/lib/castor-1.1.1.jar
<tomcat-dir>/lib/ccpp-1.0.jar
<tomcat-dir>/lib/commons-logging-1.1.1.jar
<tomcat-dir>/lib/jaxb-impl-2.1.7.jar
<tomcat-dir>/lib/log4j-1.2.15.jar
<tomcat-dir>/lib/pluto-container-2.0.0-RI.jar
<tomcat-dir>/lib/pluto-descriptor-api-2.0.0-RI.jar
<tomcat-dir>/lib/pluto-descriptor-impl-2.0.0-RI.jar
<tomcat-dir>/lib/pluto-taglib-2.0.0-RI.jar
<tomcat-dir>/lib/portlet-api-2.0.jar
<tomcat-dir>/lib/stax-1.2.0.jar
<tomcat-dir>/lib/stax-api-1.0.1.jar
<tomcat-dir>/lib/xercesImpl-2.9.1.jar
<tomcat-dir>/lib/xmlParserAPIs-2.9.1.jar
<tomcat-dir>/endorsed/jaxb-api-2.1.jar
3. Create a new directory in your domain home called jahia-deploy
4. In jahia-deploy, create the following directories and file:
1. jahia-ear
2. jahia-ear/META-INF
3. jahia-ear/META-INF/application.xml
4. jahia-ear/META-INF/weblogic-application.xml
5. jahia-ear/jahia
5. Edit jahia-ear/META-INF/application.xml and add this content:

   <?xml version="1.0" encoding="UTF-8"?>
   <!DOCTYPE application PUBLIC "-//Sun Microsystems, Inc.//DTD J2EE Application 1.3//EN" "http://java.sun.com/dtd/application_1_3.dtd">
   <application>
     <display-name>Jahia</display-name>
     <description>Jahia</description>
     <module>
       <web>
         <web-uri>jahia</web-uri>
         <context-root>/</context-root>
       </web>
      </module>
   </application>
   

6. Edit jahia-ear/META-INF/weblogic-application.xml and add this content:

   <?xml version="1.0"?>
   <weblogic-application xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.bea.com/ns/weblogic/90" xsi:schemaLocation="http://www.bea.com/ns/weblogic/90 http://www.bea.com/ns/weblogic/90/weblogic-application.xsd http://java.sun.com/xml/ns/j2ee http://java.sun.com/xml/ns/j2ee/j2ee_1_4.xsd ">
   <xml>
     <parser-factory>
       <saxparser-factory>org.apache.xerces.jaxp.SAXParserFactoryImpl</saxparser-factory>
       <document-builder-factory>org.apache.xerces.jaxp.DocumentBuilderFactoryImpl</document-builder-factory>
       <transformer-factory>org.apache.xalan.processor.TransformerFactoryImpl</transformer-factory>
     </parser-factory>
   </xml>
   <prefer-application-packages>
     <package-name>antlr.*</package-name>
   </prefer-application-packages>
   </weblogic-application>
   

7. Copy the contents of <tomcat-dir>/webapps/ROOT to jahia-deploy/jahia-ear/jahia
8. Move the following jars from jahia-deploy/jahia-ear/jahia/WEB-INF/lib to jahia-deploy/jahia-ear
   
jms-1.1.jar
jta-1.0.1B.jar
mail-1.4.jar
xalan-2.7.0.jar
9. Comment out these two sections in jahia-deploy/jahia-ear/jahia/WEB-INF/web.xml:

   <servlet>
     <servlet-name>Test</servlet-name>
     <servlet-class>org.jahia.bin.TestServlet</servlet-class>
     <load-on-startup>99</load-on-startup>
   </servlet>
   ...
   <servlet-mapping>
     <servlet-name>Test</servlet-name>
     <url-pattern>/test/*</url-pattern>
   </servlet-mapping>
   

Creating File System Directories

Jahia, by default, stores a number of items in directories inside the WAR directory, like search indexes and BLOB files. Since Jahia as a whole gets redeployed when templates get redeployed, you’ll want to change these directories to something external to the Jahia WAR:

1. Assuming that the base directory for these files is /var/jahia, create the following directories:
   1. /var/jahia/content
   2. /var/jahia/content/bigtext
   3. /var/jahia/content/tmp
   4. /var/jahia/content/slide
   5. /var/jahia/content/filemanager
   6. /var/jahia/search_indexes
   7. /var/jahia/workspaces
2. Give your WebLogic user full read and write access to these directories.
3. Edit jahia-deploy/jahia-ear/jahia/WEB-INF/etc/config/jahia.properties, and change the following properties:
   1. jahiaFilesBigTextDiskPath = /var/jahia/content/bigtext
   2. tmpContentDiskPath = /var/jahia/content/tmp
   3. slideContentDiskPath = /var/jahia/content/slide
   4. jahiaFileRepositoryDiskPath = /var/jahia/content/filemanager
4. Edit jahia-deploy/jahia-ear/jahia/WEB-INF/etc/repository/jackrabbit/repository.xml, and change the Workspaces element to this:

   <Workspaces rootPath="/var/jahia/workspaces" defaultWorkspace="default"/>
   

Deploying Jahia

Now that all of the initial configuration is complete, you can deploy Jahia to your WebLogic Server. Follow these steps for the initial deployment, and whenever you want to redeploy templates or the Jahia application proper:

1. Copy your template set web resources to jahia-deploy/jahia-ear/jahia/templates
2. Copy any compiled custom classes or resources to jahia-deploy/jahia-ear/jahia/WEB-INF/classes
3. Log in to the WebLogic Administration Console (i.e. http://localhost:7001/console)
4. If this is the first time you’ve deployed Jahia on the server:
   1. Create a new datasource with a JNDI name of ‘jdbc/jahia’ that points to the Oracle DB set up previously
   2. Deploy the jahia-deploy/jahia-ear directory as an exploded EAR application in the Deployments section
5. If you’re redeploying Jahia for template or other changes, then simply update the existing jahia application in the Deployments section, using the jahia-deploy/jahia-ear directory as an exploded EAR application.

Conclusion

Although the initial set up of Jahia for WebLogic may seem a bit daunting, running and deploying Jahia on WebLogic is just as seamless as lightweight servers such as Tomcat. WebLogic is a very good enterprise application server, and is certainly a wise choice for many organizations. And thanks to Jahia’s standards compliance, open nature, and flexibility, Jahia can be run on WebLogic with minimal configuration just like any other Java application server.

While this seems like a straight-forward question that should be easy to answer, it is actually a commonly misunderstood part of many software development efforts. With a clear understanding of quality, and by implementing concepts to assure quality within your project, you will be taking a great step towards improving your final product and increasing the return on your investment.

What is quality?

Merriam-Webster provides several definitions for quality. For the purposes of software development, there are two in particular that fit well: a degree of excellence and superiority in kind.

Don’t you want your end result to have a high degree of excellence and be a superior product? It won’t happen accidentally. Quality is deliberate, intentional, and should always be planned. Think of it as a guiding principle of your project that should be highlighted on day one. Regardless of what you’re building, quality should be one of your objectives.

What is quality assurance (QA)?

Quality assurance is the process of ensuring your deliverables meet a defined, minimum standard with the least amount of defects possible. Quality assurance requires dedication, planning, consistent attention and time.

The standards for quality assurance within your project are set during the requirements phase. These may include functional requirements, non-functional requirements, standards required for integration into other platforms, usability needs or company-specific best practices. As the foundation for your deliverables, having clear, specific requirements plays a critical role in quality assurance.

Value & Responsibility:

Quite simply, quality adds value by improving your end result. Implementing the practice of quality assurance in all phases of the project lifecycle and with all members of the project team is how to get it.

Every resource has the potential to add quality while working within their respective roles on the project. Everyone inherits this responsibility as part of the project team. Even with a resource filling the role of tester, it does not let others off the hook for maintaining quality while working on the project.

In much the same way a player on a sports team contributes to its overall success, each member of a project team is able to provide the same contribution to quality within a project. Working together as a team towards improving quality produces better results. Whether your business sponsor steps forward to clarify vagueness in a requirement, your project manager catches an error in estimates, or your developer corrects a defect prior to system testing, each person has contributed to improving the quality of the final result.

As a matter of fact, errors corrected earlier in the project lifecycle, whether they be in planning or defects in actual code, result in less overall expense to the project as a whole since they were not carried downstream. The further downstream errors and defects go, the more costly they become to correct in hours burned, resource engagement and budget consumption.

In the case of a production defect, while you can go back and correct one that’s been exposed to your users after release, you may have a hard time, or possibly never, overcome negative perceptions and opinions formed as the end user encountered the issue. The risk and potential impact to your business is significantly reduced by having quality assurance processes in place to minimize this situation from occurring to begin with.

Dedication to quality assurance will involve commitments to time, budget and resources around quality tools and practices used in the project. Include expenses for resource(s) to fill the role of tester in your project budget. Plan time for unit, integration, system, regression and user acceptance testing, both manually and automated if possible, prior to release. Provide the needed tools and environments for testing resources to verify and validate project requirements. And finally, use a system to log defects and track tasks for the project as a way to manage work and minimize risk prior to release.

Once you have the people and tools in place, commit to following through on addressing issues that impact quality. If you are doing the work to identify these issues in your product or processes, you must also follow through with getting the effort required to correct them into your project workflow to realize the value of your investment in quality.

Misconceptions surrounding quality, quality assurance and testing:

Misconceptions about quality, quality assurance and testing can easily work their way into technical groups and project teams. Be aware of the differences in some of these common misconceptions surrounding quality to keep your team and project focused on delivering high quality results.

- Quality, quality assurance, and testing are all the same thing.

While they are all related to one another, they are different. Quality is the goal, quality assurance is the process, and testing is one of the primary tools used within quality assurance to increase quality. It’s easy to lump them all together as one, and place that responsibility on one person only. By doing so, you narrow the potential for improving quality by limiting resources who can otherwise have a positive impact on your end result.

- Quality assurance guarantees perfect results.

While quality assurance processes can greatly improve the overall end result of a project, it is not a system designed to create perfection. When instituting a culture of quality amongst your project team, it is important to note that while perfection itself may be unattainable, it is not an excuse to disregard processes intended to improve quality. Higher quality typically results in higher end user satisfaction with a product.

- If everyone is focused on quality, then I don’t need a tester.

Even with everyone focused on quality, the role of a tester in software development is still one which we highly recommend be included in your project team. Acute testers have the ability to not only put themselves in the shoes of the end user, but also provide valuable details on where defects are happening and how to recreate them. This is the key to turning reported defects into implemented fixes quickly and accurately. Developers are often too familiar with the code to see some issues while end users are often not familiar enough to know if something is working as designed. Having the tester in the middle helps bridge the gap, and will help bring a higher level of confidence to your deliverables.

- With a tester on my project team, my developers will be able to work quicker since they won’t need to test.

Unit testing by developers is one of the easiest ways quality can add value to a project. When a developer finds an issue as a result of testing done as they develop, they have the ability to implement a fix much quicker than at any other time in the project. This conserves time, resources, budget and potential impact on other areas of code once integrated. Testing resources will once again verify this code as part of the whole, either directly or indirectly, during system and integration testing downstream. Unit testing will not uncover all defects in software, which is why system and regression testing is necessary as well. Unit, system and regression tests are intended to complement each other. Quality will suffer and project resources will be wasted unnecessarily if any one type of testing is used as a replacement for another.

As you get the wheels turning while contemplating the next project on the horizon, take a close look at your processes and the dynamics of your project team. Is quality a goal? Is everyone taking responsibility for quality? Will you get more value from this effort with people and processes that are more quality-centric as you work?  Odds are, you will.

New Content Management Roles

In past Jahia versions, there were a few standard roles involved in editing, consuming and publishing content:

• Content reader - anonymous users or standard logged in users who can consume content or documents
• Content editor - someone who is authorized to contribute, edit, or delete content
• Content approver - someone who is authorized to publish content to the ‘live’ site

Jahia 6.5 introduces a couple new content management roles to the web content lifecycle.  The idea is that companies have end users with varying degrees of responsibility for site design and content contribution; these new roles support this variation by getting new tools for web content contribution (these tools are discussed later).  The new roles that have been introduced include:

• Casual content editor - this would be someone who contributes content to a site, but doesn’t necessarily ‘build out’ the entire site.  For example, take an online magazine site.  A site like this has a lot of article authors who don’t necessarily care about page structures, home page banners, etc.  They need to create an article quickly and easily.
• Business site designer - this role would be someone who isn’t a developer, but needs to build sites quickly from sets of pre-defined layouts, themes, and content components.  For example, let’s say a marketing department wants to launch a microsite for a campaign.  Instead of funding an IT project to build the microsite, the marketing department has the ability to use a web-based, drag-and-drop development environment to quickly build the microsite internally, and deploy that site.

These new roles enable business units to be more agile in their web presence, and also open content management to a larger audience.  One interesting consequence of these new roles is that in many cases the traditional Jahia developer/administrator idiom could be much different; developers could be responsible for building and deploying components to be consumed by site designers instead of building templates.  Of course, companies can still choose to follow the traditional model of creating templates that provide strong structure for content editors.

Jahia Studio

Jahia Studio is a new view of content management that allows non-developers to quickly build sites in Jahia from sets of predefined components, structures, and styles.  Jahia Studio is a web-based environment for users to build sites and pages from palettes of content and social components, layouts, pages, and other content items.  If you’re familiar with graphical IDEs like Visual Studio or Eclipse with web plugins, think drag-and-drop of components from your palette to your page, only these are content components instead of user interface controls.

Jahia Studio

The introduction of Jahia Studio means that business units no longer need to engage the development team to create a site; instead the business is empowered to create sites with little training.  The development team creates an ‘inventory’ of components, layouts, themes, and the like, deploys them to Jahia, and the business can then design and create sites by consuming this inventory.  This can enable organizations to be truly agile in their web presence, by significantly reducing the time to market and reducing the inevitable gap between vision and implementation.

Casual Contribution

Casual contribution mode is a new way for content editors to contribute content, without having to use the traditional popup editors that past versions of Jahia used extensively.  Although inline editing was introduced in Jahia 6, inline editing wasn’t very useful beyond quick fixes of text content.  The value in casual contribution is that it introduces a new kind of content editor; someone who needs to contribute rich content, but doesn’t need to leverage the power tools that the full blown Jahia editor tools provide.  Building on the previous example of article authors, these authors don’t need the ability to create full site map layouts, manage access control lists, or import/export content.  Rather, they need to be able to quickly author new articles with robust editor controls in an intuitive manner.  This kind of content editing is enabled through casual contribution.

In a nutshell, casual contribution builds upon inline editing; it allows casual contributors to enter content in context of the page or content at hand.  Although traditional content editing in Jahia is very intuitive through ‘in-context’ editing, casual contribution improves the editor’s user experience and extends the audience that can contribute content.

Casual Contribution

Underneath the hood, Jahia 6.5 has augmented and/or replaced the usage of GWT with JQuery throughout the user interface.  The experience of casual contribution is just one example of how adding JQuery to the mix has greatly enhanced Jahia’s UI.

Conclusion

New content management roles, Jahia Studio, and casual contribution are just a few of the exciting new features in Jahia 6.5 xCM.  There are a plethora of other improvements and additions that we couldn’t cover here, but we hope to cover in future posts:

• Component development API
• JCR access throughout the full content model
• REST APIs for reading and manipulating content
• Improved Maven support for Jahia artifacts, like components and templates
• Ability to use multiple template sets for a single virtual site
• Improved UI and browser compatibility with JQuery
• Linking between Studio components, like linking a Rating component to an Article component

Jahia 6.5 xCM is an exciting step towards providing a true unified platform for web content delivery.  This version truly enables a much broader audience to manage web content; no longer is the management of web content limited to trained power business users and web developers.

Recently, one of our clients had a requirement to present data and content from multiple services, but didn’t want the portal look and feel.  After some more analysis, it became evident that the website needed to invoke multiple services, take the response messages from those services, and create an HTML output that incorporated the response messages as well as ‘static’ HTML, CSS, and JavaScript.  This requirement is very similar in function to a BPEL composite or a platform that implements Enterprise Integration Patterns (EIP).   In other words, service invocation is orchestrated, messages are passed and collected, and an output is created that is the composition of the messages and other data.  The biggest difference between BPEL/EIP and this requirement is that the output is HTML, not a data protocol like XML.

Given the similarities, we designed and built a web content delivery platform that leveraged EIP messaging styles but generated an HTML response.  This particular solution used Spring Integration for the messaging integration.  Other platforms that implement the same patterns could just as easily be used, such as BPEL, Oracle Service Bus, or Mule ESB, to name a few.

The basic sequence for generating web pages in our system is as follows:

Sequence Diagram of Content Integration

One of the key actors in this sequence is the page configuration; there needs to be some sort of storage of page and site configuration.  This storage needs to be able to tell the integration platform and supporting code things like:

• What services/channels do I need for content?
• What XSLT do I use for output transformation?
• What security or authentication mechanisms are required for this page?

For this particular project, we used Oracle Universal Content Management (UCM) to provide the configuration storage, as well as storing the XSLTs used to create the output.  The client had UCM installed already, and UCM has robust SOAP APIs, so it was a good fit.  In practice, any sort of persistent storage could be used, such as a relational database, Spring context configuration, and so on.  It would be helpful to use something that doesn’t require redeployment of the application (like Spring context configurations usually do), as the idea is to allow the development of sites independently of the content delivery platform.

Once the composition of the page request is determined, the integration platform is used to collect, aggregate, transform, and assemble the resultant output.  The first step is to process the request.

Using Spring Integration, a Servlet was created to handle requests for these ‘virtual’ pages.  The Java code for the doPost method looks like this:

ApplicationContext context = WebApplicationContextUtils.getWebApplicationContext(this.getServletContext());
ServiceGateway serviceGateway = (ServiceGateway) context.getBean("serviceGateway");
result = serviceGateway.getPage(requestInfo);
response.getWriter().write(result);

In the preceding code, the requestInfo variable is just a Java bean that stores a the requested page’s configuration.  The most important part of this is the page name.  In our case, the requested page is the third part of the request path; the first two are the Servlet and site respectively.  So, for a request of /serviceBus/mySite/home, the Servlet path is serviceBus, the site name is mySite, and the page is home.

The Spring context configuration for the serviceGateway bean is:

<si:gateway
  service-interface="net.avantia.bus.integration.ServiceGateway"
  id="serviceGateway"/>

And the ServiceGateway interface is a simple interface:

public interface ServiceGateway {
 @Gateway(requestChannel="page-request", replyChannel="html")
    public String getPage(RequestInfo message)
     throws MissingContentResourceException, Exception;
}

The request channel page-request is defined simply in the context configuration as:

<si:channel id="page-request"/>

None of this is very interesting though. The real key is the Message Enricher defined for the page-request channel:

<si:service-activator
  ref="bootstrapMessageEnricher"
  method="enrichMessage"
  input-channel="page-request"
  output-channel="full-request-content"/>

This bean tells Spring Integration to enrich the inbound message on the page-request channel by invoking the enrichMessage method on the bootstrapMessageEnricher bean with the supplied inbound message.  The enrichMessage method of the bootstrapMessageEnricher gets the configuration of the requested page from our configuration store (UCM in this case), and sets a number of headers in the message.  The bootstrapMessageEnricher bean’s enrichMessage method looks like this:

public Message<String> enrichMessage(Message<RequestInfo> message)  throws Exception {
  RequestInfo reqInfo = message.getPayload();
  String requestedPage = reqInfo.getRequestedPage();
  String site = reqInfo.getSiteName();
  Map queryMap = reqInfo.getQueryParams();
  String siteXmlFileName = site + "_SITE_XML";
  String siteXsltFileName = site + "_SITE_XSL";
  String siteXml = ucmContentUtil.getUcmFile(siteXmlFileName, ucmUid, ucmPwd);
  String convertedPath = convertSitePath(requestedPage);
  XPathUtil xpathUtil = new XPathUtil(siteXml);
  String pageNode = xpathUtil.getXmlFragment(convertedPath);
  if (null == pageNode) {
     throw new MissingContentResourceException("No page node found for path: '" + convertedPath + "'");
  }
  String siteXslt = ucmContentUtil.getUcmFile(siteXsltFileName, ucmUid, ucmPwd);
  String queryParamXml = wsbUtil.convertQueryParams(queryMap);
  Message<String> outMessage = MessageBuilder.withPayload(pageNode).
   setHeader("siteXml", siteXml).
   setHeader("queryParams", queryParamXml).
   setHeader("siteName", ((RequestInfo) message.getPayload()).getSiteName()).
   setHeader("requestedPage", requestedPage).setHeader("siteXslt", siteXslt).
   setHeader("servletRequest", reqInfo.getServletRequest()).
   setHeader("contextParameters", ((RequestInfo) message.getPayload()).getContextParameters().toXml()).
   build();
  return outMessage;
}

The Message Enricher takes care of figuring out the requested site, required authentication, and so on by parsing the request, finding the configuration for the requested page, and setting message headers for the requested information. The next step is to configure a Header Value Router that routes the messages to specific beans based on the message headers set by the Message Enricher:

<si:header-value-router input-channel="split-request-content" header-name="contentSrcType">
  <si:mapping value="soap" channel="soap-service-request" />
  <si:mapping value="httpGet" channel="http-get-service-request" />
  <si:mapping value="ucmGetFile" channel="ucmGetFile-request" />
  <si:mapping value="param" channel="param-request" />
  <si:mapping value="static" channel="aggregator-input" />
  <si:mapping value="http" channel="aggregator-input" />
</si:header-value-router>

The Header Value Router takes care of routing the message to various beans defined based on the header values set by the Message Enricher.  The beans defined include channels that invoke services, as well as transformers that apply XSL transformations to the messages.  The XSLTs, in this solution, generate valid HTML.

Once the routing is completed, Spring Integration sends the final message back to the serviceGateway bean, which is configured to send the message to the http reply channel.  The proxy servlet then responds to the client with the message that sits on the http reply channel.

There are a couple of design considerations that came up as part of this solution, and should certainly be evaluated before heading down this path:

1. The solution requires content to be managed by XSLT and XML: This is not a solution for non-technical end users to manage content, like a true content management system would be.  This is a pattern for aggregating content in a SOA or cloud based environment, where your web designers are savvy in XML and XSLT and need to develop a UI for service-driven content.
2. Form-driven content is not considered ‘first class’: HTML forms didn’t drive this design.  The architecture could certainly be modified to accomodate form driven interaction, but it was not a requirement for this solution.
3. Invoking multiple external services for each request impacts performance: The solution has aspect-oriented caching built into it, so that invocation results can be cached on a per-service basis.  This is a subject for subsequent posts.
4. Standard Java EE security is hard to apply to ‘virtual’ page URLs: Spring Security helps when applying custom login pages, security contexts and the like.  Spring Security was leveraged as much as possible for this solution.

In practice, this approach is better suited for delivering HTML snippets or JSON rather than full web pages.  But it is a good example of how to leverage integration techniques for more than just data message results; it gives you a way to generate user interface markup, AJAX responses, and other content markup directly from your ESB or integration platform.

Why?

How many times have you created a utility method that perhaps formats a set of numbers, let’s say a phone number, into a formatted number with parenthesis, spaces and dashes? Perhaps a zip code into a zip+four format? Not a difficult task and probably only one line of code, something like the following:

public string FormatPhone(long PhoneNumber) {
	return string.Format("{0:(000) 000-0000}", PhoneNumber);
}

Then when you want to access it, you have code similar to the following:

long PhoneNumber = 3302164400;
string Result = Utilities.FormatPhone(PhoneNumber);

It does the trick and there is nothing wrong with it. However, once you get into multiple developers working on the same project, you need to let each of them know of your great new method. Not to mention, where do you put it? Meaning, what namespace is it in? Is it generic enough to simply create a new project that is just a bunch of helpful methods like this and we reference it? If you reference it but do not add a using statement, we will still not be able to use it. This latter issue is still true with extension methods, but I am getting ahead of myself. Let’s make this same utility method into an extension method like so:

public static string Formatted(this long Source) {
	return string.Format("{0:000-000-0000}", Source.Number);
}

Programmatically this is nearly the exact same code. The only real difference is the method signature. Note that it is a static method and the addition of the “this” keyword. That last requirement to making extension methods would be that it needs to be part of a static class. That all seems logical to me since we do not want to create an instance of the class, we just want to run our extension method. So now that we have our new extension method, let’s see it in use:

long PhoneNumber = 3302164400;
string Result = PhoneNumber.Formatted();

One of the best things about extension methods is that they now are considered top-level/first class methods that were originally on the object. This means that the method will show up in intellisence when you hit the period as shown below.

Please note that in the image above, it was extending a PhoneNumber object instead of just the long, but the logic was exactly the same.

Here is another extension method that can come in quite handy and is far easier to understand (and find) than some helper method. The original code to translate a string into the current language via Sitecore is as follows:

string Result = Sitecore.Globalization.Translate.Text("Some text to translate");

Not difficult to understand at all, however, quite a lot of typing each time you need to translate something. So the following extension method can help.

public static string Translate(this string Input) {
	return Sitecore.Globalization.Translate.Text(Input);
}

Now the code to run it would be similar to the following:

string Result = "Some text to translate".Translate();

Overload

You can even overload extension methods. The following example overloads the Translate method to make it similar to the string.Format method:

public static string Translate(this string Input, params object[] Parameters) {
	List NewValues = new List();

	foreach (object Parameter in Parameters) {
		if (Parameter is string)
			NewValues.Add(((string)Parameter).Translate());
		else
			NewValues.Add(Parameter.ToString().Translate());
	} // foreach of the parameters

	return string.Format(Input.Translate(), NewValues.ToArray()).ToString();
}

This would allow you to have code like the following:

"Translate both {0} and {1}".Translate("this here", Constants.Season.SPRING);

As you can see, you can pass any valid set of strings into the parameters of the Translate extension.

Along those same lines, one of my first extension methods I ever wrote and consistently use is one that extends the StringBuilder class. A fair amount of times I am writing many lines of strings to my StringBuilder object. However, I want to format the text and have each string be on its own line. There is no existing method that both creates a new line (AppendLine) and takes formatting values (AppendFormat.) Sure, I could put an “\n” at the beginning of each line but then I have to remember that for each line and I am not a fan of having formatting code when a method can do it for me consistently. So I created the following extension method:

public static StringBuilder AppendLineFormatted(this StringBuilder Input, string Format, params object[] Parameters) {
	return Input.AppendFormat("\n" + Format, Parameters);
}

That then allows me to do the following in code:

StringBuilder Results = new StringBuilder();

Results.AppendLineFormatted("<h2>{0}</h2>", ms.Season);
Results.AppendLineFormatted("<dl>");

Results.AppendLineFormatted("<dt>{0}</dt>", s.Day.Value);
Results.AppendLineFormatted("<dd>{0:h:mm tt}</dd>", s.Time);
Results.AppendLineFormatted("<dd>{0}</dd>", s.Notes);
Results.AppendLineFormatted("</dl>");

return Results.ToString();

As you can see this is building HTML code. There are two foreach loops in the actual code that have been removed for clarity, but the above should give you an idea of what I was shooting for.

Say My Name

This brings me to another point I want to get across. I was extending the StringBuilder class that currently has Append, AppendLine and AppendFormat methods. At one point I thought about overloading the AppendLine method that takes the same formatting parameters. I tested it and it worked just as it should. However, it was doing a disservice to the extension method itself. It was not obvious that the extension method existed so it could be missed and can also make it confusing. I have read a few articles that suggest to not overload an existing method with an extension method. It makes sense and I have stuck too that. I am glad I came up with that solution on my own as well.

Is That It?

Now let’s get more complex. Currently each of the extension methods here has only been a few lines of code. There is nothing wrong with that and chances are, most of yours will be simple as well. These methods are just ways to help us save some duplicate code and typing as well as a more logical grouping and thus understanding of what is available to make our coding lives easier.

Here is an extension method that takes an Enumerable set of some generic objects that either are or inherit from a specific class. In particular, specific to a Sitecore CMS project I was working on. We had data in a separate database that the customer did not want to manage via Sitecore. We needed a way to link the mass amount of data they had with the minimal amount we needed on the Sitecore side. Using LINQ (which can have a blog post of its own, so stay tuned!) and the Entity Framework we would get back the exact set of objects we were looking for. However, we needed to now get the Sitecore data (its Link and GUID) and didn’t want to have to write similar code for each of our objects to basically do the same thing of connecting to Sitecore, find the object, match it up and grab what we need to grab. The following code shows this final method showing only two different classes, however, there were multiple we were handling.

public static void SitecoreFill(this IEnumerable Items) where T : SitecoreLink {
	Items.ToList().ForEach(item => {
		string ADDID = "";
		string ContainerID = "";
		string FieldName = "";

		switch (item.GetType().Name) {
			case "Parish":
				ContainerID = Constants.Sitecore.Containers.PARISH;
				FieldName = Constants.Sitecore.ADDIDFields.PARISH;
				ADDID = (item as Parish).ID.ToString();
				break;

			case "School":
				ContainerID = Constants.Sitecore.Containers.SCHOOL;
				FieldName = Constants.Sitecore.ADDIDFields.SCHOOL;
				ADDID = (item as School).ID.ToString();
				break;
		} // switch on the Type of object

		Item scItem = Sitecore.Context.Database.Items.GetItem(ContainerID);

		if (scItem.HasChildren) {
			Item Found = scItem.Children.Where(c => c.Fields[FieldName].Value == ADDID).FirstOrDefault();

			if (Found != null) {
				item.GUID = Found.ID.Guid;
				item.Link = Sitecore.Links.LinkManager.GetItemUrl(Found);
			} // if we found the one we are looking for
		} // if there is at least one child item
	}); // foreach of the items in the given list
}

The SitecoreLink class has two simple properties on it, the GUID and the Link properties. Now they get filled in within this extension method.

That then allows me to use LINQ to get the set of items I needed and then run this extension method similar to the following:

var Items = (from p in AdministrationQueries.GetParishes()
	where p.Name.ToLower().Contains(Criteria) ||
	p.Addresses.Any(a => a.ZipCode.Zip == Criteria ||
	a.City.ToLower().Contains(Criteria))
	select p).ToList();

Items.SitecoreFill();

Now Items has the GUID and Link properties set and it only took one line of code.

One final extension method I will pass along here is one that makes the following code a bit more readable:

string day = "Friday";
DayOfWeek dayOfWeek = (DayOfWeek)Enum.Parse(typeof(DayOfWeek), day);

That is somewhat duplicating things in my mind. Here is the extension method:

public static T ToEnum(this string value) {
	return (T)Enum.Parse(typeof(T), value);
}

Finally, here is what that would give us the ability to do:

string day = "Friday";
DayOfWeek dayOfWeek = day.ToEnum();

Or even like this:

DayOfWeek dayOfWeek = "Friday".ToEnum();

As you can see, that makes it quite a bit more obvious what we were actually doing (taking a word and converting that to a given enumeration.) This extension method flirts with a suggested rule that your extension methods shouldn’t be so generic that everything extends the string class and now there are too many to scroll through or even remember or group logically.

Trickery

One trick I have used thanks to extension methods is that sure, I have a class that holds values and does stuff. Chances are, you are going to want some sort of collection of that class. This of course could be a List or any of the other collection classes. If I wanted to write a method that dealt with a List I would have to write a new class that inherited from IList and that seems like a bit of a pain. However, with having just the single item of MyClass and then writing an extension similar to the following, I get all of the goodness of the List collection and I can extend it:

public static string DoSomething(this List Items) {
	…
}

Keep In Mind

It is important to note that just because you can write an extension method doesn’t mean you should. An example of this where extension methods may be getting abused would be one off of the string class called IsValidEmail. Sure it works, sure it is useful. However, not all strings are going to be email addresses. That should be more of a utility or helper method in some static class where you pass in the email string. In the example above with the enumeration, I say it flirts with this rule, true. However, all enumerations could be strings or at least we have all used the Enum.Parse method to convert a string to some enumeration. It always seemed like double code to tell it I am using this enumeration and then have to cast it to that enumeration. Extension Methods and how far down the rabbit hole they go may become one of those religious debates like C# vs VB or Tabs vs Spaces or Coke vs Pepsi. Time will tell.

In conclusion, I think extension methods are incredibly powerful and this may be the code geek inside me, but I think they are really quite fun. One last thing I will mention is that if you create your own objects, you can put normal methods and/or properties on them to your heart’s content. However, there may be some issues with properties and methods when you have data contracts going over the wire. Perhaps you are connecting with a third party and you do not want them to see all of your coding goodness, you only want them to see the properties. Thus putting your extension methods into a separate namespace and class breaks them out logically and allows your data contracts to flow without bounds nor worry. Also, we do not have the source code for .Net nor most third party .dll’s you may reference. However, we can still extend them as if we did have the source code, thanks to the wonderful world of extension methods.

EDA is an appropriate pattern in many types of web applications; it provides a rich user experience by telling the user what is happening, instead of the user having to request updated information.  This can be applied across many use cases:

• Portal dashboards – instead of refreshing a page to get updated dashboard information, the dashboard refreshes itself (real-time reporting).
• Social / discussion / blog – as community members contribute, followers are automatically up to date
• Business applications – business events are constantly sent to users, instead of them having to request updates.  Think of someone like a travel agent or a retailer’s customer service agent who relies on real-time data; instead of them requesting updates, the updates are sent to them as they happen.

This list goes on, but the underlying theme to modern EDA-driven applications is that they usually rely on near real-time data, are highly transactional, and are web based.  Traditional web based applications are stateless, which means they can’t respond to events on a server.  This is a departure from user experiences with thick client and client-server applications.  Technically, thick client and client-server applications are based on EDA (the user interface is built upon event handlers, not request/response models), and web based applications depart from EDA.  But the current trend is to apply EDA to web, SOA, and cloud applications, which is the point of discussion.

There are a number of ways to design EDA applications for the web, SOA, and cloud, but there’s a very compelling approach that a VMware vFabric engineer brought to my attention today using distributed data management systems designed for cloud-based applications.  Traditionally, EDA systems rely on messaging systems to raise events; there are a number of reasons for this, but one of the most significant is performance.  Remember, events are significant changes in state, which is really a change in the data for a system.

So, if a system were to react to an event, it has to be able to detect changes in its state (the application’s data).  In traditional applications, detecting changes in state (data) means constantly querying a relational database.  For high transaction systems, this means querying databases every few seconds at a minimum to detect changes, and since state is usually tied to a user, these database queries are per user.  Querying databases at this frequency just to detect changes in state (events) would crush most traditional applications, no matter how well they’re engineered.  This is why most web based EDA applications employ techniques like callbacks, asynchronous event messages, and AJAX polling from web pages.

Now, consider an application that uses a distributed data management system like vFabric GemFire.  GemFire is a lot of things, but for the sake of this discussion, it is an in-memory data layer that provides persistence, performance, and high availability.  In other words, you can query it at will without concern for performance, just like you would with in-memory objects, but still be confident that the data is persistent, as you would be with a relational database.

With GemFire, providing event notifications can be as simple as starting a thread that polls for updated data (you don’t need to worry much about the performance of constant queries); the thread, which can be resident to the application, can invoke whatever event handler is registered.  This is very similar to event handlers in thick client applications.

Taking this approach gives applications the ability to implement event polling, instead of reacting to event notifications.  If applications aggregate or integrate external systems, this is a good thing.  Instead of having to decide whether or not to respond to events, and how to respond to them, they can control which events to respond to, and what to do.

This may seem like a minute twist, but let’s extend our Twitter example to a fictitious Zapper application that provides similar functionality.  Let’s say that there are 10,000 users online.  If the Zapper backend responded to events (zaps) sent, and 1000 zaps were sent per minute, then Zapper’s servers would have to figure out which of the 10,000 users should receive updates for each zap.  That’s 10,000 evaluations per zap, or 10,000,000 evaluations per minute, that Zapper’s servers would have to process.

If Zapper was designed so that each user session polled for events (using something like GemFire in their infrastructure), then the Zapper servers have a much different profile.  Each Zapper session would poll for updated zaps frequently, let’s say every 30 seconds.  Every Zapper session would poll for updates 2 times in a minute; that’s 20,000 polls per minute (2 polls per minute x 10,000 users) compared to 10,000,000 evaluations per minute for the previous scenario.  And the polls are ultimately against an in-memory data layer, instead of sending a message to the clients that they need to re-query the relational database.

This scenario is based on a system that is highly transactional; the data is updated much more than it is read.  But systems that are built upon EDA imply that there are frequent and significant changes to the application’s data.   It is but one example of ways to design systems that rely on cloud architectures, high performance and scalability, and frequently changing data.

We will be discussing a lot of the technologies, processes, and techniques that we use every day in developing software, as well as subjects and industry trends that we are interested and involved in.  Our team is involved in a staggering amount of technology, industries, and other software development goodness, and our new blog provides a place to bring this wealth of information together.

So be sure to check back often; we have a number of subjects planned for posts in the coming weeks and months.  Some of things that we’ve been working on and will be writing about include:

• Enterprise cloud computing
• Enterprise and application security labs
• New ways of building applications, like NoSQL and JavaScript servers
• Agile business process strategies
• How to build successful teams
• And much more…

Welcome, and please let us know what you think.