Decision Model and Notation (DMN) – the new Business Rules Standard. An introduction by example.

DMN is a brand new standard of the OMG, it stands for Decision Model and Notation and is clearly related to BPMN and CMMN. DMN defines an XML format and is executable on Decision/Business Rules Engines. It is currently on the home stretch of standardization and camunda will release camunda BPM 7.4 including DMN in November. Over the last months we discussed a lot of Business Rules use cases with clients and sketched solutions in DMN. So high time to give an introduction into DMN and present some learnings we had so far.

The example: Task Assignment/Routing of new claims

The example is straight forward but very real for a lot insurance companies which currently transform their processes from manual/paper-based work into partly automated/digital form. The decision to take is:

Who (which employee) should handle a new filed claim?

Why should I care about DMN?

• Standard: DMN is not owned by a certain enterprise but by an institution (OMG), which is already established through other world-wide standards, e.g., BPMN or UML. The standard is supported by many software products; you are less dependent on any particular vendor’s products.
• Business-IT-Alignment: Decisions can be modeled and executed using the same notation. This allows to use business analysis results as “code” making changes to the rules behind the decision really easy. Given appropriate tooling it might even be realistic that business people can directly edit concrete rules within a given structure (more on this later). As rules (leading to decisions) change much more often that processes or entities, being agile in changing rules is absolutely necessary for being future ready!
• Experience: DMN was developed by people having decades of experience with Decision Management. Even though, the standard does not force any special implementation choices for rule engines, allowing for modern and lightweight implementations (like in camunda BPM :-)).

Levels for Decisions

DMN defines two levels for decisions:

• Requirements Level: What sub decisions or input data do I need in order to take the decision?
• Logic Level: Concrete decision logic expressed as decision table or by expression language.

Let’s make this concrete by looking at the so called “Decision Requirements Diagram” (DRD) for the claim assignment example:

The DRD is pretty straight forward as I used only two symbols here: Decisions and Inputs:

So you need the Inputs in order to take the decision, and the result of one decision might the input for another decision. So in our example we do a couple of decisions in a row in order to determine the employee:

• Required skill/role: We want to narrow down the number of potential employees working on the case by figuring out hard constraints, namely skills or roles. These are hard constraints in a way, that an employee not having a required role or skill is never allowed to work on the claim! The Input is the type of claim, e.g. third party liability (“I broke the mobile of my friend”) or accidents with humans involved (“I fell off the mountain when climbing – now I broke my leg”) and the information if the customer is private or business – where very different laws have to be considered.
• Exceptional case: Some insurance agents might have special agreements, like always being served by a special employee or group of people. This might even be true for big customers having huge frame contracts. Hence, Inputs are Agent and Customer – to be precise: Agent-Id and Customer-Id, but more on this later.
• Available qualified employees: The former decisions are now Input: Which skills and groups are required and which employee/group is maybe already selected by an exceptional case. Then we need a list of employees and their availabilities in order to decide which employees could possibly work on the case. Again: This is still a hard decision in the sense that it can clearly be decided for every employee if he can work on the claim or not.
• Employee appropriateness score: Now we switch to weak factors – as we want to assign the claim to the most appropriate employee. “The most appropriate” is not a clear rule but might even change often. A typical example in insurance industry are thunderstorms. After such an event the number of claims might rise dramatically for a certain region. So you might want to weaken rules like “work on claims in the region they occurred”. And you might assign more complex cases even to more unexperienced employees – as this is much better than not working on them at all as all experts are busy. The typical way to handle these weak factors are scoring models – so the first step is to score the appropriateness of every employee (by the way: we avoided the term “score employee” by intention – even if the result is a bit unhandy).
• Determine employee: The final decision is determine one single employee who “wins” the claim. This decision is easy: Take the employee with the highest score (if there are multiple employees with the same score, select randomly one of them).

The DRD helps to understand the decision and list all necessary inputs. In real-life projects the biggest effort is to gather all required inputs for a decision – as you have to query various back-end systems or maybe even introduce new data stores. In a recent workshop (with business and IT departments at one big table!) we used the DRD to go over every input and discuss how to get it. That was really helpful and productive.

But in order to understand all details of a decision and to be able to automate it we have to dive into…

The Logic Level of Decisions

Every decision node in the DRD can be defined in detail by a decision table, which is defined in DMN as well. Let’s have a look at our example giving us 4 tables:

Let’s examine the simplest of the tables in more detail:

In the table every row is one rule. The table consists of

• Input columns: Specify conditions of a rule. The columns normally relate to the Inputs you defined in the DRD earlier.
• Output columns: Define the result for a rule, if the input conditions are satisfied.
• Hit Policy: Defines if only a single rule can “fire” or if multiple rules are allowed. In both cases you can further define how you want to handle conflicts if more than one rule fires. The following lists briefly summarizes the options here:
  • Single rule can fire:
    • U(nique): Only one rule can match
    • A(ny): Multiple rules may match – must have same output
    • P(riority): Rule with highest priority (own output field containing priority) is selected
    • F(irst): First matching rule (order in table!) is selected
  • Multiple rules can fire:
    • O(utput Order): List of rules in order of priority (own output field containing priority)
    • R(ule Order): List of rules in order of table
    • C(ollect): List of all hits without order, might be combined with operator + (sum), < (min), > (max), # (count)
• Completeness Indicator: Specifies weather the rule table is
  • C(omplete): All possible input combinations are covered.
  • I(ncomplete): The table is allowed to contain “white spots”.

The decision table about the experience of an employee was a “UC” table, meaning: Single, Unique and Complete. Completely different the scoring table, it is “C+I”, meaning: Multiple, Collect Sum, Incomplete. Why is this?

The Scoring Table basically defines bonus/penalty for various situations. It is completely OK if a claim does not trigger any of these rules, then it has a score of 0. That’s why the table is incomplete. It is also OK that a claim triggers multiple rules, as you might get a penalty for too low experience and too many open claims at the same time. The penalties are collected and summed up. So in the end you will have one sum of all scores. This is exactly defined by the given table with and the “C+I” categorization.

Execute DMN with camunda BPM

The analysis of the rules so far was already very valuable. In recent workshops we discovered that this form of structuring decisions is often unknown to business analysts but helped to gather a common understanding. Surprising that it is not commonly used, as the basic method was already described years ago, e.g. in The Decision Model. But the situation is the same as it was when BPMN 2.0 came out: Even if we had workflow engines and business analysis methods already earlier, it only got widley adopted when a standard emerged, that took Business-IT-Alignment seriously and allows to save graphical models as XML file and execute them directly in an engine. So business analysis results are not doomed to reside in documentation knowbody ever reads.

DMN Support is currently build into the Open Source BPM Platform camunda BPM, so it will be able to execute DMN decision tables (as the ones shown above) in upcoming 7.4 version in November 2015. The important thing is that camunda will execute native DMN XML, there is no glitchy translation into proprietary formats necessary! Camunda tackles the whole tool chain with this first release – and of course the current state is already available in the Open Source project (as SNAPSHOT or soon as the upcoming first alpha):

• DMN Engine: The core decision engine is a small library, which works in a stateless mode. That means you just hand in a decision table as XML file, throw in a bunch of inputs (as Java objects) and get a result back. You do not need any infrastructure to use this in whatever situation you might think of.
• dmn.io: The web application which allows to build and edit DMN decision tables. It is 100% Java Script, meaning you do not need any kind of back-end to run it. It can be run standalone to edit decision tables during development time and also be embedded into other applications easily.
• BPM Platform Integration: By integration the DMN engine into the BPM platform all features of the overall BPM platform are leveraged, meaning you have a persistent repository of DMN rules which can be versioned. That allows you to just reference a decision table by name to take decisions. The same mechanism as with BPMN will apply: If you build “Process Applications” containing DMN decision tables they are automatically deployed. Decision definitions can be inspected in Camunda Cockpit. There, dmn.io will also be hooked in to allow live editing decision tables by operating or business stuff.

If you are interested in the current development status best checkout the projects at GitHub:

• DMN Engine
• dmn.io

Changing Rules easily

As mentioned one of the main advantages of using DMN is Business-IT-Alignment and the chance that business people can directly change rules. Wait a second – doesn’t this sound like the “Zero Code Lie“?

It is important to look at the right things now. Personally I think it is completely realistic that a non-IT-person can edit the concrete rules – meaning the rows in a decision table. You could insert new rows, delete some or simply adjust values. This is exactly why an insurance company wants to have this thing: If the thunderstorm is there, you can easily decide that the region is now unimportant for the next 5 days (or the like). You might even decide that some skills are not required, which sacrifices the quality of decisions for speed of processing – which might be worth it (which is a business decision!).

It is unrealistic that the very same person changes the structure of the rule table and cares about the binding with the environment (like input data types). That should still be prepared by proper IT people.

Interesting observations can be made when watching discussions about going-live of changed rules, approval workflows, responsibility of changes and so on. But thats enough content for a blog post on its own some time in future.

Non Decision Table Decisions

You hopefully wonder how that decision from above is now really executed. And you might have recognized that I did not present a decision table for every decision node. The reason is that the remaining two decisions does not fit in a decision table. Let’s have a look at them in more details.

Determine Employee: As already mentioned, this is a simple “use employee with highest score” decision. Nothing we need a table for. DMN defines an expression language called FEEL (Business “Friendly Enough Expression Language”) which could handle this. But it could be also implemented in the surrounding process easily (shown later) which is my preferred option.

Available qualified employees: This is a tough decision. Looking back at the DRD you might even get a feeling that this does not really fit in the DRD. Do you? I do! The problem is that this decision handles a different multiplicity. So far we only looked at one object – our claim – which we got as input. Now we want to get a list of qualified employees. That raises two questions:

• Do I have to load each and every employee in order to hand it into the decision engine?
• How do I handle the “multiple instance” problem that from this decision node on I have to handle multiple objects (one claim + n employees)?

Unfortunately a decision definition in DMN cannot (yet?) handle the different multiplicities. So strictly speaking the DRD I presented earlier is wrong! The problem is visualized here:

We hit a typical real-life problem which can be solved by a method I call the “decision funnel”. We are currently discussing how to tackle this in DMN internally. For the moment the workaround is to hook in the decisions in an overall BPMN process, which we normally have to do anyway in order to execute the service calls to gather input data. This is shown in a minute.

The Decision Funnel

If you have rules that pick a certain entity (like the employee) you have to respect certain non-functional requirements. For example it might not be a good idea to load all employees of a big company just to “rule out” 99% of them because they are not allowed to do a certain task (like handling the claim). It might be a much better idea to:

• 1. Decide about requirements/criteria.
• 2a. Load all employees fitting the criteria (by a service call or simple SQL).
• 2b. Maybe DO some ruling out by DMN, for example for exceptional cases or to get the best trade-off between performance and easily changeable rules.
• 3. Score all found employees.
• 4. Select the employee with highest score.

Implementing the Decision Funnel using BPMN + DMN

The following BPMN process implements that funnel for the claim assignment. With BPMN we have order activities and Multiple Instance semantic at hand:

The decision tables presented above can be hooked into the BPMN Business Rule tasks:

Executing the full example

What do you have to do in order to get this example executed on camunda BPM? This is rather simple:

• Save decision tables as DMN XML file.
• Add Bindings, meaning that you clearly define for every column how this is mapped to input/output data objects (Java). In camunda BPMN at this moment JUEL (Java Unified Expression Language) can be used here.
• Call Rule Engine from Business Rule Task. This will be a built-in functionality soon – for my early prototypes I used a piece of Java code.

Let’s have a very short look at how the XML file looks like to get some basic understanding. I marked one rule bold – so you can follow the binding:

<?xml version="1.0" encoding="UTF-8"?>
<Definitions xmlns="http://www.omg.org/spec/DMN/20130901"
   id="definitions" name="camunda" namespace="http://camunda.org/dmn">   
   <Decision id="decision" name="CheckOrder">
      <DecisionTable id="experienceOfEmployee" name="Experience of Employee" isComplete="true" isConsistent="true">
         <clause id="input1" name="Approval Authority">
            <inputExpression id="inputExpression1" name="Approval Authority">
               <text>employee.approvalAuthority</text>
            </inputExpression>
            <inputEntry id="inputEntry1">
               <text><![CDATA[< 1000]]></text>
            </inputEntry>
            <inputEntry id="inputEntry2">
               <text>[1000..10000]</text>
            </inputEntry>
            <inputEntry id="inputEntry3">
               <text><![CDATA[> 10000]]></text>
            </inputEntry>
         </clause>
         <clause id="output1" name="Experience" camunda:name="experience">
            <outputEntry id="outputEntry1">
               <text>low</text>
            </outputEntry>
            <outputEntry id="outputEntry2">
               <text>medium</text>
            </outputEntry>
            <outputEntry id="outputEntry3">
               <text>high</text>
            </outputEntry>
         </clause>
         <rule id="rule1">
            <condition>inputEntry1</condition>
            <conclusion>outputEntry1</conclusion>
         </rule>
         <rule id="rule2">
            <condition>inputEntry2</condition>
            <conclusion>outputEntry2</conclusion>
         </rule>
         <rule id="rule3">
            <condition>inputEntry3</condition>
            <conclusion>outputEntry3</conclusion>
         </rule>
      </DecisionTable>
   </Decision>
</Definitions>

That basically means that in order for rule1 to fire we evaluate the expression #{employee.approvalAuthority > 1000}, and when this is true we set the result experience = low. For anybody interested in more details: I will push the complete source code to GitHub soon – let me know (bernd.ruecker@camunda.com) if you are eager to get that code!

Outlook

So far the first steps with DMN are great! I like the method and I like having an executable standard. And I already love the lightweight implementation in camunda BPM (thanks to the awesome camunda BPM team!) allowing to execute the decisions really easy. Sure – it is still an alpha stage – but to quote one customer: “Even that pre-alpha version works better than [some other existing rule engine product we tried].” 🙂

Personally I will try to add more blog posts the next weeks with more examples or my thoughts on DMN + CMMN. Stay tuned!

By the way – we are collecting tons of material, examples and learnings to distill into a formidable DMN training course (not yet announced in English – will soon follow). Make sure you don’t miss the first date – it will for sure be a lot of fun!