Abstracting the Process Engine
Wow, what unexpected moves in the area of jBPM recently! Starting with Tom and Joram leaving the jBPM core team (see Die Zukunft von jBPM) to develop a new engine (see Alive and kicking). And yesterday – after being quite for a pretty long time – JBoss gives their answer for the future of jBPM and a vision for jBPM 5 (see Announcing jBPM 5). But what does that mean for projects using jBPM already or plan to start using it? Hence a pretty common question from customers approaching us is: Which engine and which version should we use?
And that is a tough question. Even with some more insights of the projects I find it hard to answer. Let’s have a quick look at the situation:
- jBPM 3.x is supported by Red Hat and successfully used in a lot of projects, hence it is pretty stable. We have a lot of experience in that area and know every bit and byte of it. But it is not really developed further.
- jBPM 4.x is not yet finished and some bugs or missing feature might catch you in the project. Operating it in a larger scale is not yet proven (as far as I know). And no Support from Red Hat is available.
- Drools Flow is in some intermediate state as well. No support is offered by Red Hat for it.
- jBPM 5 is till now just a wiki page. The concrete Roadmap is not yet defined.
- The new engine from Tom and Joram is not yet released and it is not yet sure when it will be stable and usable in production exactly.
- And there are even some other engines like Bonita or the like.
Technically I think the differences of the engines are obviously existent, but not that huge. As written in my last blog: Finally it is “just” a state machine 😉 For the decision which engine to use I think you should have a look at other arguments, for example:
- Is there professional support available?
- Will there be maintenance for the engine?
- Does the open source project have a vital community?
- Is it proven in bigger real live projects?
- Is there a vision of the project?
So for example one of our customers starts a new project at the moment. For him the decision for the engine is pretty hard. First thing we now decided is to create a good abstraction of the engine before plugging it into the project. This is something I have done or seen pretty often already, actually I would say in most of the projects. I am not sure why we just needed the engine confusion to get the obvious idea:
We want to create a Process Engine Abstraction Layer to make the Process Engine pluggable. That’s a bit like slf4j or JDBC for Process Engines. And it will be Open Source, independent of a concrete engine and available for everybody!
Abstracting the Process Engine
This idea was in the JBoss PVM already, with the difference that it was pretty much part of the jBPM project. What we want to do is to define the API on our own. Then we can provide a binding to jBPM 4 as well as to the new engine, jBPM 5, or whatever. Ah, and another customer has another problem: He uses jBPM 3, and even if he is pretty happy with it, he needs to switch to BPMN 2.0 later on at least for political reasons. So there we want to provide a binding to jBPM 3 to allow an easy migration path from jBPM 3 to another engine in 2 steps: First change the code to use the abstract API and if that runs without problems you could switch to another engine smoothly. Even you could implement to run both engine in parallel to avoid runtime process instance migration. Or to put it in easy words: Already running process instances finish in the old engine, new process instances are started in the new engine. And that will be handled transparently behind the scenes 🙂
And how to migrate the process models you may ask? Bloody good question! Our approach is to use a mapping to a business model in BPMN. This model is independent of the engine and could even be mapped in another format later on. Sounds unrealistic? Maybe in a generic way, but currently we already used it successfully in a pilot project, so it looks promising!
I will provide a draft of that Abstraction Layer / API as soon as possible, but I want to discuss that internally in camunda and with a couple of customers before going public. But don’t hesitate to contact me if you are interested in such a layer: bernd.ruecker@camunda.com. What I can say is that it (surprise, surprise) will not miles away from the jBPM 4 API. But I currently have a look at other API’s and engines as well, e.g. jBPM 3, Apache ODE, an upcoming BPMN 2.0 Engine and custom build abstractions from various projects.
So in a prefect future you can answer the question “Which engine do you use in your project?” with “it doesn’t matter” 😉 So stay tuned, it will get a really interesting summer!
Cheers
Bernd
P.S: Today I wrote in English, since I think these thoughts are interesting in an international context. Sorry for our German speaking fans 😉
We started using jBPM over a year ago at first as a proof of concept. At first 3.2.x but then we jumped onto 4.x which offered a much better API and better design on the DB side.
I personally think that the fact that the key jBPM man, the “father” Tom Baeyens left this project and what is even worse will start a new competing product is the last call to everyone out there who has direct dependency on jBPM in their mission-critical enterprise projects.
This announcent has only strengthened the criticizers of Open Source projects in large companies. And at this particular occasion I have to agree with them that using jBPM especially after what happenned would be a high risk of staying with a product which might die in a couple of years.
Only two viable options exist in this situation: a) go for a commerclal potentially very expensive product like WebSphere Process Server; or b) use an abstraction layer like proposed by Bernd.
Unfortunately both options are not without caveats. Whereas the first one implies clearly high license costs, the second one is also potentially risky as you can never expect to develop the perfect abstraction layer catering for all your needs. You will have a considerable development, testing, maintenance costs as well.
We went for option b) and now feel it was the right decision to stay jBPM independant.
This option has an advantage that we can replace jBPM with other Workflow Engine just by writing a new adapter.
Hi Henry.
Thanks for you comments! I don’t like the situation for the same reasons of “slapping jBPM 4.x projects in the face” and especially for JBoss not making a statement about jBPM 4.x future. Agreed.
On the other hand I think your option a (buying a commercial product) doesn’t free you of that risk. I saw big vendors stopping a product for the sake of another new one they maybe recently bought. Making strategic decisions based on the latest buzzwords to increase sales. So I am not convinced that things are really better on this side.
And as I already wrote on several occasions: There are use cases (like embedding the engine in the own software) which cannot be satisfied by the big commercial players out there…
Thanks for your comment on your abstraction layer (as written, most of the projects did or tried to do it). For this matter I will write you an additional mail 🙂
Cheerio
Bernd
I agree with Bernd: with commercial products I argue the risk of incompatibility between versions is even higher than with open source.
The abstraction layer is the only sensible approach. You have to factor it in the integration costs of the solution, or count it as technical risk/debt if you use directly the product’s API.
People come and go all the time, so it’s not really a big deal, even if Tom was the main brain behind jBPM. The people at Drools are already reorganizing to support the jBPM use case, so there’s future ahead.
Now, the important thing is maintaining focus, close jBPM 4.4 and get it out of the door, and gathering support and feedback around 5.
Execute me. Michele. I don’t think it is a good choice to kill jBPM 4 and throw it out of the door. I believe the drools is a amazing project, but it still need time to study and make familiar with it. Futhermore there are already lots of projects have used jBPM. How could you let these projects move to drools from jBPM smoothly?
HI Bernd,
if you are going to program your own API: did you think about using a standard Workflow API like that of WfMC?
Or should I send the API of our system. Then you may use SAPERION Workflow in the case jBoss may be out 😉
And for the BPMN model: the way from signavio into SAPERION will be ready as well as discussed here:
http://www.saperionblog.com/lang/de/saperion-und-signavio-auf-dem-weg-zum-bpm-round-trip-engineering/1220
Martin
Hi Bernd,
I agree with you that differences between BPM engines, specially those written in Java, are not that important so your idea of defining an abstraction layer to work with different engines really makes sense.
On that topic, Charles Souillard, Bonita Open Solution project lead has just released a trilogy of articles illustrating how Bonita APIs can be leveraged in a Java application: http://www.bonitasoft.org/blog/category/tutorial/.
I really think that main differentiator between BPM Solutions is not the engine anymore but the graphical environment to define, run and manage processes.
best regards,
Miguel
@Bernd, I understand that you’re attracted with building a generic bridge for framework as you did it a couple of times in projects for good reasons. But I’ld like to provide a counter perspective.
I think if a company starts using a BPM engine, then it is a good idea to build a clear separation and isolation around the engine. The main point here is that the isolation is build only for the use cases that this project uses from that particular engine. It defines a subset of the engine’s capabilities that you use. In that respect it helps the company to keep the dependency on the BPM engine organized.
If then later the engine needs to be replaced, then you this company own interface is only focussed on the use case for that company. So they know quickly which methods they should be looking for in the new engine and can estimate the feasibility.
If you try to build such a bridge in a generic way, I believe this is too complex. I agree that in essence, they’re all just state machines underneath. But abstracting the API for the state machine + all the features around it can easily be underestimated.
@Henry and all other jBPM 4 users,
It was beyond my control and I regret very much that jBPM 4 users are left people in the cold. Especially I regret the bad vibes this might show on open source in general. That’s why we’re now putting a lot of emphasis of making our new initiative a multi company collaboration effort. So that it’s not dependent on one company.
@Miguel, I agree that graphic environment and tooling is important. But I propose that we agree to disagree on the engine capabilities. I think there are a lot of features that make a difference in the core engine as well. If we want to bring BPM for the masses, we need embeddability (in any java tx environment, incl the cloud-without-db-with-eventual-consistency) and pluggability (of activities and process languages). I’m sure we can settle this dispute soon with a Tempranillo when we meet next time. Looking forward to it 😉
for those who’re interested, here’s jboss’ official response: https://community.jboss.org/blogs/mark.little/2010/04/23/the-future-of-workflow-in-jboss
regards, tom.
Just to be clear: Yes, Tom Baeyens has left the jBPM team, and we wish him good luck in his next challenge. No, RedHat is not dropping jBPM or any of its users, it has stated that it will continue to support the jBPM project. For those that are interested, there’s even a proposal for jBPM5 out already:
http://community.jboss.org/wiki/jBPM5RequestforComments
So, yes, there are some risks when using open-source, but this is exactly what RedHat is trying to improve: it will provide support for specific version so that you’re not left alone in the cold, no matter what happens, based on your support contract. And we’ll try to offer you a migration path to newer versions if you want that as well. And our doors are still open, we always welcome feedback and collaboration, so if you’re interested in BPMN2 (or BPM) in general and want to try out BPMN2 or even collaborate with us on the engine, just contact anyone on the jBPM team.
Anyway, I believe standardization could be an important factor in this entire process. This seems to be happening with BPMN2 on the process definition format, but there doesn’t seem to be consensus on other areas like various APIs, history logging, etc. Only time will tell if we’ll ever standardize all these as well, but I believe standardization is definitely a driver for open-source. We at least are definitely committed to offer the jBPM users the native BPMN2 engine and tooling around it they’ve been promised, and much more.
Okay, finally I want to answer some comments, sorry for the long delay…
@Tom: I don’t agree with your view here. I agree that it is not easy to define such an API, make it powerful enough, allow extensions and so on (extension mechanism is something we are in discussions at the moment). But the good thing for us is, that we can work together with our customers to define that layer. So it will be even easier for Users of that API to adapt to a new engine, because the binding doesn’t have to be developed on their own 🙂 And in terms of risk mitigation, it is the best strategy these days…
Additionally, doing it project specific unfortunately leads to immature APIs pretty easy, I have seen that often. I don’t want to blame anybody for it, normally you have to build it pretty early when you don’t have experience with the Engine yet.
@Henry, Tom & Kris: Okay, I don’t want to push this blog entry discussion in the direction if jBPM has a future and/or if JBoss reacted badly. I hope, jBPM has a future as much as anybody else! It was an amazing product in the past already (especially the simplicity of jBPM 3 and the clean API of jBPM 4). But anything else should better be discussed on the jBPM forum. Let’s see where the journey leads us to…
@Miguel: I have a different view on it as well. Tom’s point are valid on the one side, and on the other side in our projects more and more time is invested on modeling conventions, Business-IT-Alignment, SOA related problems (like correlation) and tooling around all of that. There we currently try to put some of our experience and ideas into open source tooling as well (I will blog about that pretty soon)… So I think there is still enough to do in every area of BPM 🙂
[…] Suite. Wir selbst werden natürlich auch ein Binding unserer Abstraktions-API bereitstellen (siehe Abstracting the Process Engine). Zusätzlich wurde übrigens bei der Architektur von vornherein darauf geachtet, dass sie auch in […]
Bernd,
Thanks for using English… American’s like myself are monolingual, probably a mild form of retardation from eating too many Big Macs.
Process Manager APIs are near and dear to my heart… As you know, in many enterprise solutions there will always be products from multiple vendors, and many of those products will have their own embedded process managers.
Clients want visibility into their entire process, not just that part managed by a specific Process Manager. We want happy clients, so we need standardized Process Manager APIs… Your effort is as good a place to start as any.
-John
[…] liberate Apache license may boost usage possibilities. In camunda fox remain some special features: PEAL – The Process Engine Abstraction Layer (doesn’t make sense to have an abstraction layer inside an engine) and BPMN to jBPM 3 […]
[…] have to exchange the Engine in your environment (which maybe is not that hard, if you used our Process Engine Abstraction Layer […]
[…] by changing the code to use Activiti’s API. Alternatively, you may also think about using a wrapper for the API of the process engine, if you anticipate future migrations and want to abstract from the process […]