jbpm deployer for jbpm in enterprise environments

After writing the jBPM meets ESB article I want to write today a bit more in depth about that integration. In a current big project at a customer we are involved to set up the environment including JBoss ESB and JBoss jBPM. If you really want to use these technologies in bigger environments you sometimes feel like the first poor guy doing it 😉 So I want to share some of my problems and solutions with you here.

Today I write about classloading issues with jBPM in the context of versioning and scoped deployments.

Problem

What we want to achive is a working environment based on JBoss AS, ESB and jBPM. Mixed in the soup we need to use JTA transactions and scoped Classloading because of different versions deployed in parallel. An overall requirement is being extremely scalable and robust, because the company will have a huge amount of transactions on their systems. As a side issue a concept for deploying and correct versioning of jBPM processes should be developed. Correct versioning means aligned with the application and service versioning in place.

I want to present different aspects we tackled and our current solution, which I called the jbpm.deployer.

Versioning

We make a difference between versions and patches:

• New versions of an application or service are not compatible with the old one. So the old version must continue to run in parallel to the new one. Clients specify which version they use and keep using the old one until they explicitly update to the new one (cause this may involve changes, testing and so on).
• Patches are compatible fixes of one version. The patched code replaces the original code. The changes have to be transparent to the client.

So how to deal with that?

First of all the application versioning is solved by adding the version to the EAR name. By doing so, the different versions can be deployed in parallel on a JBoss application server (by switching on scoped classloading, for more information refer to the JBoss Wiki). Basically with EJBs this boils down to have different JNDI-Names for different versions of enterprise beans and a common helper class being responsible to find the right reference for you. This is basically the same for ESB deployment artefacts, but having different service names, not JNDI names.

The question is how jBPM process versioning (see jbpm user docs for details) can fit into that logic? There may be in fact different approaches, what we did is to add the version number to the process definition name as well (so the process name gets for example “OrderProcess_1.0”. A new version (not patch) of a process becomes a completely new process definition in jBPM. The built in versioning of jBPM is used only when deploying patches. If a processdefinition.xml gets patched, a new jbpm process version is deployed to the jBPM database.

This is illustrated in the following picture:

Versioning with ESB and jBPM

Note that if you have a patch in the process definition, running process instances keep running in their versions! If you want to migrate them to the new version this can be done by writing some “migration script”, which can be implemented by a jBPM Command (see Command ApiDoc), a good starting point is the ChangeProcessInstanceVersionCommand. This can be (more or less) easily extended to maybe take Groovy scripts for process migration or anything else that make sense in your environment.

Implications on classloading

The versioning described above has some implications on classloading:

Writing Java classes to the jBPM database, which is possible if you add them to your process archives (.par, see jbpm user doc for details) like described in the jbpm user doc, does not make sense in our situation, because patches normally replace the existing code base. And really new versions resist in different deployment artefacts, so you can have different versions of the same Java class.

This is a good thing, since in my opinion Java code should be placed in “normal” deployment artefacts (EAR, JAR or the like) and put on the “normal” classpath as often as possible. Only in some rare cases you still want to write code to the database (and then you can still do it).

But now we get to the very interesting part. Let me add one additional “background” information: jBPM should obviously run “just once” in our application server (as shown in the picture above). We do not want to package jBPM inside of every EAR or ESB archive we deploy, and basically this would also not work in every situation. And for monitoring and management we want to be able to connect to this central jBPM instance. For example the web-console shipped with jBPM could be used to look at process instances and maybe trigger one of them (trigger means sending a signal to the token in the jBPM vocabulary). Now: How can this central console load classes which resist in different EARs or ESB packages?

The solution we built in order to achieve this builds upon the ProcessClassLoader used by jBPM. This classloader implements the functionality to load classes from the database for a specific process definition (in a specific version). If the classes are not found in the database, it delegates to the parent class loader. Starting from jBPM 3.3 (scheduled for November 2008) this class loader is configurable via the jbpm.cfg.xml (see Jira: JBPM-1148). One additional side note: A problem with it was, that this ProcessClassLoader was not set as ContextClassLoader when executing delegation classes (actions, decisions or the like), I fixed this as well for jBPM 3.3, see Jira: JBPM-1448).

Own ProcessClassLoader and ClassLoader registry

For the solution we build our own ProcessClassLoader, loading classes not only from the database but also from the right EARs. As mentioned we can configure jBPM to use it, so there is no need for any dirty hacks. But wait, how do we know which are the “right EARs” for classes? To answer this question I want to show briefly how packaging works in our environment. As you can see in the next picture, process archives (see jbpm user docs for details), containing the processdefinition.xml along with the image and maybe Java classes (but hopefully not as stated above), are put inside of an ESB or EAR archive (the described solution works for both cases even if we normally just use ESB archives). Classes required by the jbpm process have to be in the ESB package too, if they are not on the app server classpath (e.g. lib directory).

Packaging for Deployment

When deploying the ESB archive we know that we have to use the ESB classloader for the process definition contained in the process archive. If the same process definition (better: a process definition with the same name) exists in different ESB archives, it has to be a different version as already explained. Hence we could write an own JBoss deployer, listening for process archives (.par files with a processdefinition.xml in it), picking them up and doing 2 things:

• Deploy the process definition to the jBPM database if necessary (means either the process definition does not exist in the database yet or the content has changed),
• Register the ClassLoader of the current deployment unit (e.g. the ESB or EAR) in a central in-memory registry.

During runtime, our own ProcessClassLoader can check this ClassLoader registry in order to find the right ClassLoader for a process definition. The following picture illustrate the whole idea:

The big picture of the jbpm deployer

Implementation details

Basically the so called “jbpm.deployer” is not very complicated. The deployer itself is a MBean, so what you need is a jboss-service.xml:

<?xml version="1.0" encoding="ISO-8859-1"?>
<server>
  <mbean
   code="...JbpmDeployer"
   name="jboss.esb:service=JbpmDeployer">
     <depends>jboss.jca:service=DataSourceBinding,name=YOUR_JBPM_DS</depends>
  </mbean>
</server>

I want to explain more insides on the code for the jBPM deployer and ProcessClassLoader here as soon as possible, so stay tuned…

The whole deployer is just a directory on your jboss application server, which contains this MBean, the jbpm libraries and the jbpm configuration.

If you are working with the JBoss ESB, obviously the jbpm libraries and configuration has to be removed from the jbpm service there, what remains is just a queue and the JBpmCallbackService:

The required configuration in jbpm (jbpm.cfg.xml) is again easy, just add:

<string name="jbpm.classloader" value="context" />
<bean   name="jbpm.processClassLoader"
        class="...ProcessClassLoaderFactory" singleton="true" />

Basically that’s it. By the way, besides all that we use the existing jbpm-enterprise.ear (but without any jbpm libraries or configurations, so we definitely have them only once on our system). With using the enterprise version of jBPM JMS is used for asynchronous Jobs and EJB-Timers for jBPM timers, so we switched of the JbpmJobExecutor completly.

What we gained from it

With the described solution we

• can have one central jBPM database, even for monitoring and administration. There we can even trigger processes using classes in different EARs or ESBs,
• neither have to include the jbpm-jpdl.jar in every EAR/ESB nor having to put application classes in the JBoss lib,
• can use scoped deployment to deploy different versions of your applications (ESBs/EARs),
• allow jBPM processes using classes from these EARs (action classes, interfaces to remote beans, …),
• don’t have to deploy Java classes to the jBPM database,
• shouldn’t have any more classloading issues (concerned with jBPM).

And I also see advantages for possible future improvements. For example you could add interceptors or the like to the deployer which maybe check for global conventions or whatever…

In the project I mentioned the solution seems to work pretty well, even if some tests still have to be made…

Feedback desired

Please provide feedback! Either for my proposal or maybe as well in form of experience reports how you solved this or similar problem! I am also in touch with the jBPM and ESB team to maybe commit back some of the ideas to the projects itself, but therefor we need more information and a better understanding of the general use cases…

Just send me an email to bernd.ruecker@camunda.com. Thanks and have fun with the JBoss SOA Platform! And don’t give up, after solving the first trouble, it is really a cool technical basis 🙂