Computerwoche is one of the most read magazines for IT-Professionals in Germany (I would actually assume it is THE most read one), and therefore I was quite happy I could contribute an article about BPM-Software to their last edition. Well, the article is in German, of course, but if you are brave enough you will download the PDF and try to read it anyway If not, here comes the core message of it: I divided BPM-Systems into the categories “pure play”, “embedded”, “saas” and “open source”. I consider that useful, although there can be overlappings such as an open source – BPMS that is embedded into a commercial product like an CRM or ERP. Then I thought about the future of commercial pure play BPMS and came to the result, that a shift of market shares is likeley, boosting the relevance of both Open Source BPM and SaaS BPM at the expense of “classical” on premise, closed source BPM products, which was also one of the reasons for our engagement in the Activiti project.

One of the main reasons behind that shift is that Open Source gives you more control over your technical BPM-Infrastructure and therefore less vendor-lock-in, especially if the OSS Stack does not require your Engineers to learn proprietary ways of implementing process applications, but let them do all that with their Java skillset plus BPMN (which is a standard and therefore not vendor specific). This makes OSS the perfect choice for supporting processes that implement your core competences and strategic USPs, while you can go for out-of-the-box SaaS-Solutions for processes that are “just” supporting your core processes, e.g. in HR, Accounting, Helpdesk or similar things (well, depends on your company, of course…). However, that OSS based BPM-strategy works only out if you do have a strong Java team in your company, no doubt about that. Well, I did not say that EVERY company is heading that way, but I am expecting or actually experiencing quite a few companies doing that already, e.g. based on JBoss jBPM.

So, the bottom line is: OSS BPM is not necessarily cheaper, but it makes you more independant, if you fulfill the prerequisites (<= qualified Java Resources).

Screenshot of the Cycle GUI

On the left you see a first screenshot of the Activiti Cycle web application. There you have a tree view of repositories. The basic idea is, that you can access different types of repositories here in one unified tree. For the first release we concentrated on file system, Activiti Modeler and Signavio Modeler Repository. But actually our main goal was to create a good API for repositories, so that we can add other repositories later on easily, e.g. SVN. And you could even do that on your own for whatever you need. In the screenshot you can see that I included multiple SaaS-Signavio-Accounts, possible as well For the next iteration we plan some thinking on tags and virtual folders, to structure that tree more easily even if the amount of repositories and folders is growing.

When clicking on one item – we call it artifact – you can see details on the right. These details differ depending on the type of artifact. A BPMN model maybe wants to show the diagram, for Powerpoint slides you may want to see a preview, XML code may need some highlighting and so on. We call these kind of detail views content representation, and an artifact can have multiple ones, in the example to the left you can see, that a BPMN model currently has 4 representations: The image, the Signavio JSON sourcecode, the BPMN 2.0 XML and some BPM 2.0 XML made more readable for the developer.

Action with form to create technical model in other repository

Comparision between raw and developer friendly BPMN

The camunda fox jBPM 3 plugin is automatically integrated

Process Models can be validated for executability

Okay, overall the current features may not seem overwhelming, but as I already mentioned now we have a good, extensible foundation. Even if we scheduled some bigger refactorings for the next iteration, I think we are on the right track. And once we have this basis stable, we can do the really fancy and cool stuff. And we will The release of Activiti 5.0.beta1 is on Wednesday September 1st, so I can just recommend to download it and try it out yourself! And don’t forget to give us feedback… But maybe don’t dig too deep into the code yet, as some stuff is still subject to change.

But actually things have changed a bit. After announcing fox, I had a longer discussion with Tom Baeyens, the project lead of Activiti and the former project lead of JBoss jBPM. We know each other since I think over 5 years now (wow, time passes!). And we figured out we had pretty similar visions in mind. Tom and me agreed, that it would be a perfect synergy, if we put the energy and ideas planned for camunda fox into Activiti Cycle, the component of the Activiti BPM platform designed for collaboration, development process and Business-IT-Alignment. Hence we took over the lead for this component and were busy the last months to migrate existing code to Activiti and get started there.

But now I am pretty happy with this situation. We can concentrate completely on the fox/Cycle ideas and everything is embedded in the quite creative Activiti Team. And the best is: With the coming Activiti 5.0.beta1 on September 1st, we can ship the first preview of Activiti Cycle! So this is not camunda fox 1.0, but at least we keep our promise to ship And actually the whole migration and learning curve for new technologies like Spring SURF for the GUI took us the time, in which we would have done fox

First Cycle Screenshot with fox jBPM Plugin

But it was worth the investment, Activiti Cycle is embedded in the Activiti BPM platform, but not tied to it. Actually we already have a working project with using Activiti Cycle together with a jBPM 3 Engine . And the very liberate Apache license may boost usage possibilities. In camunda fox remain some special features: PEAL – The Process Engine Abstraction Layer (doesn’t make sense to have an abstraction layer inside an engine) and BPMN to jBPM 3 transformations (some information on that here), which use LGPL based code that cannot be included in Activiti. But the nice thing is, the camunda fox features can be easily used as Plugins in Activiti Cycle (see Screenshot), cool eh? Currently I am writing a separate blog post to show some first Screenshots and Features, stay tuned for it!

Activiti Cycle Team

So long I want to thank the Activiti Cycle Team at camunda to make the beta1 release happen, even if it involved some night shifts. And thanks to the whole Activiti Team for support! Finally we could proudly present Activiti Cycle in time. So now we have the foundation, let’s change history with it Or maybe I should get some sleep first…

It took some time, but now you can find the official “BPMN 2.0 by Example”-Document on BPMN.org. The direct link is

http://www.omg.org/cgi-bin/doc?dtc/10-06-02

Besides our example concerning the incident management process that I have already blogged about, it contains a small introduction into the basic principles of working with BPMN (Chapter 5: Small examples introducing core concepts).

On top of that there are quite interesting examples about the serialization of BPMN-diagrams and the according diagram interchange between BPMN modelers, and more complex diagrams are demonstrated, like the famous Email-voting example.

I consider the document rather useful for getting a first but profound impression of BPMN.

But if you want to get an easy to read guide about how to apply BPMN with loads of best practices based on massive project experiences, AND you are looking for the perfect book to improve your German, hang on until the middle of september and get the second edition freshly from Amazon.de .

More Information: http://www.bpmn2010.org

See you there?

Recently we announced http://fox.camunda.com/ and I already talked about it on a couple of conferences or similar occasions. Now I had the honor to write on infoQ about it and the experiences we made so far. Basically I tried to create a plot in the various things we do there at the moment and give a better impression on where we are heading. Hopefully that makes it easier for you to catch my ideas, easier than searching through our blog and presentations Since today you can find the article online here:

http://www.infoq.com/articles/collaborativeBPM.

Enjoy! I am curious about feedback… And stay tuned, we plan to release something soon.

Stephen White has already blogged about it: BPMN 2.0 is finished, approved by OMG architecture board and will be published soon. For camunda it was mainly Falko Menge who represented us in the FTF, and he did a very good job (thanks Falko!). Falko and me also wrote a good part of the “BPMN 2.0 by Example”-Document (many thanks to Ivana Trockovitch, Denis Gagné and the other authors!), that should work as a tutorial for understanding the basic principles of BPMN. Because the whole package is not published officially yet, I just want to blog one of the two chapters we wrote that I consider interesting for people who want to get from “business-friendly simple diagrams” to “directly executable XML”.

At some points I will also add some comments from our project experiences that of course could not make it into the official OMG document.

Chapter 6: Incident Management

In this chapter we want to show the different perspectives you can take on the same business process, using BPMN. In the first step we will provide a rather simple, easy to read diagram that shows an incident process from a high level point of view. Later on we refine this model by moving from orchestration to collaboration and choreography. In the last step we take the organizational collaboration and imagine how a process engine could drive part of the process by user task assignments. The main purpose of this chapter is to demonstrate how you can use BPMN for creating simple and rather abstract diagrams, but also detailed views on human collaboration and finally for technical specifications for process execution.

Creating process models for both business AND it is actually one of the absolutely main topics of our consulting business. And it is a very big struggle, of course. For us it was important to show in the document that BPMN is not necessarily “too complicated for business”, because it totally depends on how you actually use the standard when process modeling. That’s why we always need a Framework around BPMN if we want to apply it in bigger modeling engagements.

6.1 High level model for quick understanding

Figure 6.1: Incident management from high level point of view

This diagram is really simple and somehow a “happy path”, because we assume that we always find a solution we can finally explain to the customer. The model lacks all details of collaboration between the involved employees, and the abstract tasks indicate that we do not have any information about whether the process or parts of it are executable by a process engine. This diagram is useful, if you want to scope the process, get a basic understanding of the flow and talk about the main steps, but not if you want to dig into the details for discussing process improvements or even software driven support of the process.

In our Framework, we call that kind of diagrams “strategic process diagram”, or sometimes “logical-abstract process diagram”. You could say that this way of process modeling is according to the Process Modeling Conformance sub-class “Descriptive” defined in the BPMN 2.0 spec. We do this kind of modeling very often in As-Is-Workshops and early stages of a process improvement project. And we often do not use software tools for it, but white board drawing and sometimes our home-made BPMN magnets.
The diagrams in the examples document however are all made with Trisotech’s Visio-based BPMN Modeler, provided for this purpose by Denis Gagné. The cool thing is that we could directly serialize the diagrams into BPMN 2.0 XML with that tool

6.2 Detailed Collaboration and Choreography

Figure 6.2: Incident Management as detailed collaboration

Figure 6.3: Incident Management as choreography

Some BPMN experts consider collaboration diagrams, i.e. the modeling of different process participants in different pools, only as a pattern for business-to-business-collaboration. Well, you can use collaboration diagrams that way, but their use is definetely not limited to that area. In fact, especially in companies that are NOT process-orientied, it can be very enlightening to model the collaboration between departments,teams or even roles (as we did here) explicetely with pools and message flows. And it can be the first step towards engineering the requirements for a workflow management – solution, as shown in the next paragraph.

6.3 Human-driven vs. system-driven control flows

Figure 6.4: Incident Management with human-driven and system-driven pools

The same idea applies for the software developer: Let us assume the 2nd level support agent sits in the same room as the developers. Maybe it is more efficient if the support agent just walks over to the developer and talks about the issue, rather than playing some time consuming ping-pong-game with task assignments. Therefore, we want to keep this part of the incident management human driven as well: no process engine driving the collaboration between 2nd level support and software developers. But we do want the assignment of tickets to 1st and 2nd level support agents by a trouble ticket system, which now takes the role of the process engine and therefore is modeled in a dedicated pool.

That system can actually receive and parse emails sent by the account manager and opens a ticket for it. If the 1st level support agent decides that this is a 2nd level issue, he does so by documenting his decision and completing the assigned task “edit 1st level ticket”. The trouble ticket system then routes the ticket to the 2nd level support agent.

When that agent has finished, he maybe declared the issue to be fixed in the next software release. Then the trouble ticket system makes a service call on the product backlog system, a new feature we have introduced with our process engine: The entry does not have to be inserted manually any more. In the end, the trouble ticket system will send an email to the account manager, containing the results of the incident management, and close the ticket. The account manager can then explain the solution to the customer based on the information in the ticket.

Figure 6.6: This is the only part of the whole collaboration we will execute in a process engine

Separating the “human control flow” from the “engine control flow” is actually the heart of our modeling approach, when aligning the process models of business and IT. How can we consistently visualize that some routing decisions, i.e. Gateways such as XOR, AND and Event, are taken by the process engine, and others by the people working in the process? We can’t, as long as we model both of them in one pool! For a single pool, there can be only one driving force, that’s why it is totally irrelevant in what lane you put a gateway (a fact BPMN beginners often don’t know and hardly understand), the gateway will be interpreted by the single driving force behind the pool anyway. In the end, there will be only one pool in this diagram that is actually executed by a process engine, the other ones will be “executed” by the people working in the process (Account Manager, Support Agents, Developer). We have also created an executable BPMN 2.0 XML of the process engine pool for the examples document.

Figure 6.5: This rather simple diagram is all we have to show to the account manager

Further comments (not part of the examples document)

Basically, the whole approach goes like this:

• Make a “strategic” process diagram (Figure 6.1), just a simple sketch for a quick understanding
• Make an “operational” process diagram (Figure 6.2) for analyzing the collaborational aspects
• Enrich the diagram with the aspects of a process engine, therefore adding a pool for the process engine
• Take that process engine pool into your technical environment and enrich it for execution (make a “technical” process diagram)., see also here and here.

Defining Views in Signavio

Furthermore, we are currently implementing the whole tool chain for Business and IT based on open source components in our project camunda fox, so it’s worth to stay tuned.

Test Scenarios

In a current project we develop an executable process. Hence, this process is a “piece of software”, which need to be tested. In software development this is common sense, so there are enough tools and best practices. Is that the same for technical process models? Hmm, not that easy. On the one hand we can use proved Java Test frameworks like JUnit for Open Source Process Engines like jBPM. On the other hand processes have some specialties, which aren’t surprising but somehow challenging…

For the process we devloped process scenarios, which indicate on which path a process instance flows through the process. The picture on the left shows a foto from the current project. By the way: The green mark shows: This scneario was successfull tested Sorry for the bad quaility of the picture, but otherwiese I couldn’t have posted it here.

To implement such a scenario we can write a unit test, which set up some test data, trigger the process and waits, till the process arrives in a defined state. As soon as the process waits for an external trigger (like a Human enter some data or a system sends an asychronous answer) the unit tests comes into action and “fakes” that answer. By doing so, the whole process can be walked through to check if the behaviour is as expected. Some sample code is shown at the very end of this blog post, I don’t want to frighten non IT folks too early

To define these kinds of scenarios is pretty typical by the way, we did that in quite some projects already. The downside is at the moment, that there isn’t much good tooling for that. In the area of Open Source Process Engines there exists basically nothing and the commercial tools tend to be too GUI centered, which normally isn’t developers friendly. I see the following requirements for such a tooling:

• We want to organize these scenarios! We want to mark them in the real process model, save it and load it later on.
• I want to compare the audit data from the really executed process with the defined scnerios (”Did the process really went that way?”). I want to recognize differences easily and graphically.
• I want to create “smaller” scenarios for just parts of the process. In our example process we have different parts, e.g. data checking, customer creation, order provisioning, …. We want to test these parts independantly from each other (I want to spare the explanattion why we didn’t create subprocesses for them). So we want to start a scenario at an arbritrary node in the process, provide some input data and stop it at a defined node where we calidate the outgoing data.
• Ideally Input and Output data are provided in a non technical format, like spreadsheets or Fitnesse.

Test Coverage

There is another interessting aspect of process testing: The so called Test coverage. The idea is easy: I want to make sure that every node and path in my processes was executed at least once. If you need every possible combination can be an endless discussion, but in my experience it would be already a big improvement for most projects if they had at least executed every path once. In our project I used a marker to indicate which parts we already tested as you can see indicated on the left. But this coverage report could be generated by the tooling itself, so to visualize this automatically would be another item on my wish list.

Currently we implement the mentioned use cases in camunda fox for JBoss jBPM 3.2.x and Signavio. The visualizations are done in the BPMN model in Signavio, the technical implementation and testing is done in jBPM. But we try to achieve that non technical folks can define, refine or at least read the test scenarios as well. Then we could make the tests and test coverage part of a process cockpit or process status as well.

Process Development Status Indication

By the way: Currently we use a printed BPMN diagram to indicate the status if the development, as you can see on the left. We want to link the nodes with JIRA-Issues, so we can create that indication automatically. Even if paper and whiteboards are good tools, that only works if the whole team is in the same room or at least building. Obviously it is nice to stand in front of the whiteboard for discusssions, bit not having it electronically has some disadvantages:

• Not transportable: We could’t easily take the status and scenarios with us into meetings.
• Changes: If there are process changes, we would have to reprint and redraw all scenarios.
• Keep it up-to-date: After the project or in the next project we want to improve the scenarios or change them according to changed processes. The paper from the last project is normally not longer available at that time.
• No Continous Integration based on paper
• Distributed teams: As soon as your not together at the same physical location any more you need electronical help to get that running and the collaboration intact.

But this is enough content for an own blog post…

Okay, but I promised to give you some more technical details, so I present an example test case here (note that the asserts are hidden in the ProcessHelper);

public void scenario2() throws Exception {
  testDataHelper.setCustomerAddressType(CustomerAddressType.VALID_ADDRESS);
  ... // set some more data for testcase

  long orderId = startOrderProcess();

  // provide a test stub for one service called on the way we wanted to "mock" out
  processHelper.setTestStubResult(CheckAddressAdapter.class, createCheckAddressResultResult());

  processHelper.waitForState("Fraud Check aufrufen");
  // "fake" result of fraud service: Fraud OK
  correlationFacade.callCorrelationService("Fraud", orderId, "0");

  processHelper.waitForState("Bonität prüfen");
  // "fake" result of boni service: Boni yellow
  correlationFacade.callCorrelationService("Boni", orderId, "1");

  // wait for the process to end
  processHelper.waitForState("Ende", 70);
}

During the execution the ProcessHelper shows the flow of the process, we “just” need to add some functionality to validate this audit data aginst our expectation in the scenario:

SCENARIO: Storno Boni (yellow); ORDER ID: 4368835; ROOT TOKEN ID: 1495
   - wait for Call Fraud Check
      - status changed to BI: Insert Data
      - status changed to DCN: Insert Data into Duplicate Checker
      - status changed to CCD: Create customer
      ...
      - status changed to Call Fraud Check
   - wait for Check Boni
     ...
      - status changed to Check Boni
   - wait for End
      - status changed to Check Boni
      - ...
      - status changed to End