Keep informed?
Subscribe for our newsletter now!

Business Process Management (BPM) is a very famous buzzword today. Applied correctly, BPM can lead to better architectures and more flexible software systems. But when new processes are designed or existing processes shall be improved, it is always hard to predict the resulting performance.

Business Process Simulation (BPS) is of great help in such estimations by using statistical methods to gain a better understanding of runtime behavior. But good BPS tools are rare and costly, making simulation uninteresting for a lot of companies. Because of closed source and closed documentation the tools are seldom useful in research, too.

The main objective of an already finished master thesis was to create an open source BPS tool on top of JBoss jBPM, an open source business process engine. The simulation can answer what-if questions concerning resource numbers or input parameters as well as compare different versions of processes regarding special performance indicators, for example costs or performance. Statistical input data can be taken from audit log data whenever possible. Altogether the tool can simulate completely new processes as well as support continuous improvement.

A tutorial with a simple showcase and a real life prototype demonstrates that the simulation works and how it can be configured and used.

There are several artefacts of interest available:

Master Thesis

The research about this topic and the development of the tool were done by a master thesis:

Partner

These were three partners of this master thesis:

Introduction

Simulation is not a new topic in computer science. Especially in the field of logistics or production planning it has played an important role for some years now. An example could be the simulation of a harbor loading dock’s utilization. Because it is quite expensive to correct planning errors after constructing the dock, a good modeling is a crucial success factor. And due to the random nature of events it is hard to develop an analytical formula for the result under different circumstances, that’s why simulation is used.

In the last years, Business Process Management (BPM) became very popular. This maybe surprising at first, because focusing on business processes is not new, actually it got very famous in 1993, when Hammer and Champy introduced the Business Process Reengineering. But because of the evolution of the affected tools, the situation has changed. In the beginning, process descriptions were normally “paper ware”, which means documentation only. In most cases today, process descriptions are real models, which have some graphical representation, but can also be interpreted automatically. This increases the value of the process descriptions and makes it interesting for a wide range of companies. Additionally, the current hype around Service-Oriented Architectures (SOA) pushes the development further towards BPM.

When simulation of business processes emerged, these projects included a high effort for capturing statistical data from your running business to enable valuable simulations. This situation is changing, because of the automation of business processes with so called Business Process Engines. These engines do not only execute the process model, but also collect a lot of audit data during execution, which is a good starting point for simulation.

But most of the currently available simulation tools concentrate only on pure simulation, without putting any focus on business processes. Thus you need a special model for simulation on the one hand and have to provide figures from external sources on the other hand, for example via spreadsheet tables. The simulation can answer questions concerning the resulting behavior, compare versions or show impact on changing figures. Even if this approach has the advantage, that nearly everything can be modeled and simulated, the downside is the increased effort on the dedicated model you need to build.

A better approach is to integrate the simulation tool in the BPM environment, so it can share the same model with the process engine. This safes effort when modeling a business process and enables accessing historical data of the engine for later optimizations. But there are only few tools available in this area without big shortcomings. And as these tools are very expensive they are normally overkill for small or midrange companies. What is missing is a simulation environment, which is lightweight in the terms of operation and costs. This leverages business process simulation to a much wider range of people, for example smaller companies or universities.

A good opportunity to make it easily available is the more and more accepted open source software. One existing Open Source BPM tool, meeting the requirement to be easy to use and cost-effective, is JBoss jBPM, which is already used in some bigger real life projects. The company JBoss has achieved a good reputation, even in the field of mission critical business software. For this reasons, I consider JBoss jBPM as a good foundation for the simulation environment developed during this thesis.

Summarized the main motivation behind this thesis is to develop a complete BPS environment based on open source tools. With available source code, a good documentation and tutorials on hand, this environment enables a wider range of companies to leverage, a wider range of technicals to learn and a wider range of scientists to study or research BPS concepts.

Simulation language is under discussion

The simulation language / jPDL langauge enhancements presented here were the first proposal from me. Neither the names nor the features itself must be up to date. I kept this article because it conatines some ideas and the motivation why some of the things are like they are.

The example process, instrumented for simulation

For this first example, I created a very easy process. Because I just want to demonstrate the language features, it comes without a business story behind it. The process definition XML file, with the simulation enhancements, is shown below.

The process flow is very easy, all nodes are in a simple sequence. Task one and two require some human (of swimlane ‘tester’) to do some work, the state represents some automated work, which only requires two ‘big machines’ to be done. So far, so easy. Now lets image you have to plan the resource requirements for this scenario. What information do you need for this?

First you need to know, how much processe instances will be started in which time interval. Secondly you need information about the time needed for finishing the tasks and the automated state. Imagining uniformly distributed process start times and constant processing times, you could calculate the resource requirements very easy with a pocket calculator. Unfortunately, the world is much more random, so you know that these figures aren’t the real truth. So you have to map the randomness into fitting distributions for the required input figures (how this is be done exactly is skipped for the moment). Let’s assume you want to use ‘normal distributions’ for the processing time, but stick to a constant process instance starting time.

Translating these information to the simulation language and including it into the process definition looks like:

<process-definition name='FirstSteps'
    start-distribution='start new process instances of test'>
  <distribution name='start new process instances of test' sample-type='real'
                type='constant' value='20' />
  <distribution name='time required for task one'          sample-type='real'
                type='normal'   mean='25' standardDeviation='10' />
  <distribution name='time required for task two'          sample-type='real'
                type='normal'   mean='6'  standardDeviation='1'  />
  <distribution name='time required for automated state'   sample-type='real'
                type='normal'   mean='6'  standardDeviation='1'  />       

  <resource-pool name='big machine' pool-size='3' />
   
  <swimlane name='tester' pool-size='2' />

  <start-state name='start'>
    <transition to='task one' />
  </start-state>
     
  <task-node name='task one'>
    <task swimlane='tester' time-distribution='time required for task one' />
    <transition to='task two' />
  </task-node>

  <task-node name='task two'>
    <task swimlane='tester' time-distribution='time required for task two' />
    <transition to='automated state' />
  </task-node>

  <state name='automated state' time-distribution='time required for automated state'>
    <resource-needed pool='big machine' amount='2' />
    <transition to='end' />
  </state>

  <end-state name='end'/>

</process-definition>

The bold elements are newly introduced for simulation:

What’s missing

Some concepts / language features are still missing here, I will introduce them later in an own article. Basically missing at the moment are:

The simulation run

The presented process, instrumented with additonal informations for simulation, can be executed now by the jbpm simulation environment. Please note, that we concentrate on resource utilization / resources required only at the moment, but there could be also other goals! I present the results of an example run here, but different runs can differ in the results if other seeds are used, because of the randomness of the input distributions.  The DESMOJ framework, used for the simulation part, can produce some basic HTML report with the most important performance indicators of our model. Let’s have a look at it:

Tallies

Title

(Re)set

Obs

Mean

Std.Dev

Min

Max

Waiting before Task(task one) 20.0 374 0.57607 1.74716 0.0 10.69206
Waiting before Task(task two) 55.70188485291133 373 0.83769 2.8193 0.0 25.68006
Waiting before State(automated state) 61.55679511504111 373 0.49633 1.48047 0.0 10.64398
Cycle times for ProcessDefinition(test) 67.86379347461589 373 39.63271 13.16527 11.0 84.0

Queues

Title

Qorder

(Re)set

Obs

QLimit

Qmax

Qnow

Qavg.

Zeros

avg.Wait

refus.

resource pool queue for ‘tester’ FIFO 0.0 111 unlimit. 2 0 0.07039 0 4.75592 0
resource pool queue for ‘big machine’ FIFO 0.0 55 unlimit. 2 0 0.02468 0 3.366 0

Distributions

Title

(Re)set

Obs

Type

Parameter 1

Parameter 2

Seed

time required for task one 0.0 374 Normal 25.0 10.0 91080577
time required for task two 0.0 373 Normal 6.0 1.0 43946618
time required for automated state 0.0 373 Normal 6.0 1.0 44944377
start new process instances of test 0.0 375 Real Constant 20.0 71529105

Short comment on simulation results

What you can see at the moment are waiting time before the nodes, queue sizes for the resources (where process instances have to queue up, if there is no resource available), cycle times of process instances and information about the input distributions.

Look at ‘task two’ for example: We had a average waiting time of 0.83 there (let’s assume hour as the unit for the moment), but also a maximum waiting time of more than 25 hours! The minimum of zero waiting time is clear because I ignored the warmup phase in this example, so the first run never has waiting times. This possible bottleneck in our process is somehow obvious if you look at the input distributions: Every 20 hours one process is started, but task one takes around 25 hours. Even with 2 people beeing tester, we get a problem (remember: tester is also used for task two).  The average waiting time of process instances for a tester to get available is 4.75 hours.

The average cycle time of our process in this experiment is 40, so waybe we should improve our staffing strategy here 😉

The DESMOJ GUI

Just as a short addition here, I want to include a screenshot of the DESMOJ graphical user interface, even if I don’t think it will be used later in the jbpm simulation environment. Basically the main feature, not included in the command line startup, is the possibility to show the usage of resources over time graphicaly:

Actions & Scripts

In jPDL processes you can use Actions or Scripts to execute usercode during process execution, normally Java code. Often in simulation runs, you don’t want to execute that usercode, because maybe it is not a good idea to change customer data in a real life ERP system. Also your simulation runs may include large numbers of started process instances and the usercode is really slow and don’t add any useful information to the simulation runs or influences the process execution path. Even if the problem is in practice slightly different, it can basically be compared to the testing of processes.

So we need the possibility to influence the execution of the usercode during simulation, the following snipped from a jPDL process definition shows the mechanism:

 <node name="normal actions">
  <transition to="special simulation actions">
    <action name="action 1" class="...TestAction">
      <shouldBeExecuted>false</shouldBeExecuted>
    </action>
    <action simulation='skip' name="action 2" class="...TestAction">
      <shouldBeExecuted>false</shouldBeExecuted>
    </action>
    <action simulation='execute' name="action 3" class="...TestAction">
      <shouldBeExecuted>true</shouldBeExecuted>
    </action>

    <script name ='not executed in sim 1'>
       <expression>executed.add(\"script 1\");</expression>
    </script>
    <script name ='not executed in sim 2' simulation='skip'>
       <expression>executed.add(\"script 2\");</expression>
    </script>
    <script name ='executed in sim 1' simulation='execute'>
       <expression>executed.add(\"script 3\");</expression>
    </script>
  </transition>
</node>

<node name="special simulation actions">
  <transition to="task">
    <action simulation-class='...SimTestAction' name="sim-action 1" class="...TestAction">
      <shouldBeExecuted>false</shouldBeExecuted>
    </action>
    <simulation-action name='sim-action 2' class='...SimTestAction' />
    
    <script name ='simulation script 1'>
       <expression>executed.add(\"script 4\");</expression>
       <simulation-expression>executed.add(\"sim-script 1\");</simulation-expression>
    </script>
    <simulation-script name ='simulation script 2'>
       <expression>executed.add(\"sim-script 2\");</expression>
    </simulation-script>      
  </transition>
</node>

Scripts without any configuration for simulation are ignored in simulation runs, this can be achieved by specifying the simulation=skip attribute too. If you want to include a action or script with the default behavior, you have to specify the attribute simulation=execute.

Another possibility is, to specify an own action class/script expression for simulation. In this case, only the special behavior is executed.

The last possibility is, to specify simulation-action or simulation-script elements. These are configured like “normal” ones, but are only executed in simulation runs.

ActionHandler

Besides using the language features to influence user code execution, you can also use a programatic concept: Interfaces. For this purposes, I have defined two relevant Interfaces:

These interfaces can be preferred, if the different behavior during simulation is motivated by implementation reasons. If it is are business motivated decision to change the behavior, it should better be included in the process definition.

Please note: Even if a action is not configured with ‘simulation=skip’, the simExecute method is executed, if your ActionHandler implements the SimulationHandler interface.

Probabilities of outgoing transition

In many cases, the simulation framework has to simulate decisions made by humans, foreign systems or even by jBPM itself. To configure this, you have to specify probabilities for outgoing transitions to give the simulation engine a hint, which transition to chose. The probabilities can be configured for the following elements:

The following part of a process definition give examples of all cases:

<!-- needed to schedule an end event for the task & state, constant time of 1 -->
<distribution name='no work'    sample-type='real' type='constant' value='1' /> 

<task-node name="task">
  <task name='task' time-distribution='no work' />
  <transition to="invalid end-state" name="invalid" probability='0.0'/>
  <transition to="state" name="to state"            probability='1.0'/>
</task-node>

<state name="state" time-distribution='no work'>
  <transition to="invalid end-state" name="invalid"/>
  <transition to="unaffected decision" name="possibility two" probability='1.0' />
  <transition to="unaffected decision" name="possibilty one"  probability='1.0'/>
</state>

<decision name="unaffected decision"  expression='valid'>
  <transition to="invalid end-state" name="invalid" />
  <transition to="simulated decision" name="valid" />
</decision>

<decision name="simulated decision" expression='invalid'>
  <transition to="invalid end-state" name="invalid" />
  <transition to="end-state1" name="possibility one" probability='0.2' />
  <transition to="end-state2" name="possibility two" probability='0.7' />
</decision>

As you can see, probabilities can be chosen freely. Internally all specified values are summed up and build the upper bound of the used uniform distribution. For every node an own distribution is created, to keep them statistically independant.

Email-Nodes & Timers

Email-Nodes and Timers are currently completly skipped, even if they are present in the process definition. For Email-Nodes this means, that they behave like a “naked” node.

Open Issues so far

The simulation language presented yesterday and today enables you already to specify simple simulation runs for one process. It is still under discussion (see thread in the JBoss jBPM developer forum) but should not change dramatically. Currently, I have already written basic unit tests for all features (and got a green bar :-)), but there are still a lot of open issues, for example

Some of these issues will be implemented later in the ongoing master thesis, some will be skipped for later. Or for volunteers 🙂

My next step is, besided adding more unit tests, to develop a show case for a short, but real business story. This demo should show not only how the simulation works, but also give some motivation, why it makes sense to apply it.

Experiments & Scenarios

I want to define the two basic terms experiment and scenario first to avoid any confusion at this point. A scenario defines one special setting for a simulation run, including all configurations for resource pools, distributions and so on. The scenario is identified by a name, like “low resources on christmas time”.

One or more simulation scenarios form the experiment. Run or simulation time configurations are made for this whole experiment. It can contain one or more scenarios. If you have more than one, the scenarios are simulated seperately and may be compared afterwords.

Please note, that an Experiment in the simulation framework DESMO-J, which is used as a basis in the jbpm simulation, corresponds to one simulation run, which is a scenario in our terms. But because DESMO-J stays under the hood, I don’t expect any confusion for simulation users here.

The configuration XML

The experiment is configured in an XML file. The easiest way to specify scenarios is to inline them in this file. I just give an example of the current proposal here and will write an XSD later, if the XML gets fixed.

<experiment name='MySimulationExperiment'
          run-time='100'
          real-start-time='30.03.1980 00:00:00:000'
          time-unit='minute'>
  <scenario name='NormalStaffing'>  

    <distribution name='new start dist'
       sample-type='real' type='constant' value='20' />
    <distribution name='new task dist'
       sample-type='real' type='normal'   mean='25' standardDeviation='10' />
    <distribution name='time required for task two'
       sample-type='real' type='constant' mean='10'/>

    <resource-pool name='tester' pool-size='2' />      

    <sim-process name='test' path='Test/processdefinition.xml'>
      <task-overwrite task-name='task one'
                      time-distribution='new task dist' />
      <process-overwrite start-distribution='new start dist' />
    </sim-process> 

  </scenario>
  <output path='./' />

</experiment>

Configurations made here overwrite everything which may be configured in the processdefinition.xml already. Actually it should be possible with this overwrite mechanism to specify the whole simulation configuration in this external file and leave the processdefinition.xml untouched. However, I think the most convenient way is to specify some defaults in the processdefinition.xml and only overwrite what you need here. So if you want to try different staffing strategies for instance, just overwrite the resource-pools.

I plan to support to configure scenarios in own files, but this is just on the todo list for the moment. Same with the idea of “reusable building blocks” in the configuration, like for example a block of all start process distributions named “christmas time”, “summer holidays” or “normal”.

Running an experiment

Running the experiment, means executing the simulation scenarios, is done by reading a JbpmSimulationExperiment from the XML file (with the ExperimentReader) and call the run method. This can be done easily in your own Java code, but it is also planned to support that somehow directly in Eclipse later.

Simulation result

All simulation statistics get picked up after each scenario was executed automaticlly. So you can obtain a SimulationResult object for every scenario from the experiment afterwards. This object includes all statistical data in pure POJO’s, here you can see what information you have in these classes:

ValueStatistics are for example waiting times (of tokens) before a resource gets available or process cycle times. QueueStatistics tell a lot about resource pools.

Not only resource costs matter

Scenarios of different processes differ often not only in the cost for resources, but also other values which may be expressed in money, for example

A lot of things on the other hand are hard to express with money, because they are based more on feelings. A good example for the second category is the damage of your image because of delayed orders.

Include business figures in simulation

I want to have a look on the computable figures for now, because the idea is to include them into simulation scenarios and get a better comparability. Normally the figures should depend on process variables (e.g. order profit), so I see two easy possibilities to support this:

Both is a straight forward approach which fits quite natural in the jBPM philosophy. Honestly I have to confess, that I think there can be more sophisticated possibilities for the purpose. But on the other hand, this approach is easy to implement for the moment and can be used to gain experience and maybe get better ideas.

Here is an example of what I have in mind

<scenario name = "slow invoice payment">
...
  <business-figure name = "missed discount"
       type = "costs"
       automatic-calculation = "none|process-start|process-end"
       handler = "org.jbpm.sim.tutorial.TutorialBusinessFigureCalculator " />
  <business-figure name = "missed discount"
       type = "costs"
       automatic-calculation = "none|process-start|process-end"
       expression = "#{order.discountForFastPayment}" />
...
</ scenario >

The simulation can now calculate the “costs” you have because your payment process was too slow for getting the early-bird discount. The result of the expression has to be some Java number.

The second option I mentioned above is to implement a Java class doing the cost calculation. Therefor I created the interface org.jbpm.sim.kpi.BusinessFigureCalculator. You can implement this easily, in the one and only method you have access to the whole execution context of the process instance running, so you can do nearly everything.

The type attribute specifies how the business figure is evaluated, I think of three types:

If none is selected you have to specify, where the calculation has to be done. This can be either configured in the experiment as override:

<sim-process name='test'>
   <node-overwrite node-name='node1'>
     <calculate-business-figure name='testFigure' />
   </node-overwrite>
</sim-process>

or you can add a simulation action to the desired place in the process-definition file, where the calculation should be executed. This action has to look like this:

<simulation-action class='org.jbpm.sim.kpi.BusinessFigureAction'>
   <name>missed discount</name>
</simulation-action>

If the type of the business figure is “costs”, it is automatically added to the overall costs for a scenario at the end. The other cost types can be queried seperatly from the ScenarioReport.

Generate or change process variables during simulation (Data source and data filters)

You may have asked yourself, from where the order objects in the example above some from in simulation. Very good question! The solution I propose for this is quite easy, you need some generator for process variables. At the moment I see two general possibilities for that:

Both approaches are supported by the simulation environment, I created the interface org.jbpm.sim.variables.ProcessVariableSource for this purpose. One implementation which ships out of the box is the HistorialProcessVariableSource, which uses a jBPM command to get historical objects and provides them one after another. If the number of historical data is exhausted it starts again from the beginning.

If you want to use a more sophisticated approach or your own source for variables, just implement it and hook it into your scenario. The xml configuration for it looks like

<scenario name="slow invoice payment">
  <data-source name="orders"
     handler="org.jbpm.sim.variables.HistorialProcessVariableSource">
     <name>order</name>
     <process>invoice payment</process>
  </data-source>

or if you provide your own implementation

<scenario name="slow invoice payment">
  <data-source name="orders"
    handler="org.jbpm.sim.tutorial.TutorialProcessVariableSource" />

If the data source does not send any more data (indicated by the hasNext() method), the simulation will be stopped. This enables you to execute terminating simulations.

A second concept simulates the change of data, which would normally be done by some human using a gui or result data from a foreign system call. I called this data filters and I have provided the org.jbpm.sim.tutorial.TutorialProcessVariableFilter interface for this purpose.

<data-filter name="orders"
   handler="org.jbpm.sim.tutorial.TutorialProcessVariableFilter" />

The usage of the data source and filter is easy now, you can either use it for the start event or in task-nodes or states:

 <sim-process path="datasource/processdefinition.xml">
  <process-overwrite start-distribution="start">
    <use-data-source name="orders" />    
  </process-overwrite>
  <task-overwrite task-name="change return order"
                  time-distribution="change return order">
    <use-data-filter name="orders" />    
  </task-overwrite>
</sim-process>

If you want to use it directly in the process definition you have to add a simulation actionrefering to the right data source or filter:

<event type='node-enter'>
  <action class='org.jbpm.sim.datasource.UseDataSourceAction'>
    <name>orders</name>
  </action>
</event>
<event type='node-enter'>
  <action class='org.jbpm.sim.datasource.UseDataFilterAction'>
    <name>orders</name>
  </action>
</event>

Warmup phase and steady state

In Simulation you have so called warm up phases. In this warm up phase, statistical measures are biased, for example because of empty queues at the beginning. But the most simulation runs reach a so called steady state after some time. For more information please have a look in my master thesis on this topic.

To get unbiased statistics you should reset all statistical counters at the beginning of the steady state. This is supported by the jbpm simulation tool via configuration parameter:

<experiment name='MySimulationExperiment'
          run-time='100'
          reset-time='50'
          time-unit='minute'>
...

Please note that the detection of the time the steady state starts is currently left to the user.