A little while ago, I ran into a question in Quora that hit me in the stomach… figuratively, of course. Someone asked “why do rules engines fail to gain mass adoption?”. I had mixed feelings about it. In one hand, I am very proud of our decision management industry, and how robust and sophisticated our rules engines have become. In the other hand, I must admit that I see tons of projects not using this technology that would help them so much. I took a little time to reflect on the actual roadblocks to rules engines.
A couple of points I want to stress first
Evangelization
In addition to the points I make below, with a little more time to think about it, I think it boils down to evangelization. We, in the industry, have not been doing a good job educating the masses about the value of the technology, and its ease of use. We rarely get visibility up the CxO level. Business rules is never one of the top 10 challenges of executives, though it might be in disguise. We need to do a better job.
I’m signing up, with my colleagues, for an active webinar series, so that we can address one of the roadblocks to rules engines, and decision management!
Rules are so important, they are already part of platforms
The other key aspect to keep in mind is that business rules are so important in systems that they often become a de-facto component of the ecosystem. Business rules might be used under the form of BPM rules or other customization, but not called out as a usage for rules engines. Many platforms will claim they include business rules. The capability might be there, though it may not be as rich as a decision management system. Many vertical platforms like Equifax’s InterConnect platform include a full-blown decision management system though. When decision makers have to allocate the budget for technology, this becomes another of the roadblocks to rules engines, as they assume that rules are covered by the platform. Sometimes they are right, often not.
Rules in code is not a good idea
Let me stress once more that burying your rules into code or SQL procedures is not a good idea. It is one of the roadblocks to rules engines excuse we have heard probably the most. I explain down below that this is tempting for software developers to go back to their comfort zone. This is not sustainable. This is not flexible. We did that many decades ago as part of Cobol system, mostly because decision management systems did not exist back then. We suffered with maintenance of these beasts. In many occurrences, the maintenance was so painful that we had to patch the logic with pre-processing or post-processing to avoid touching the code. We have learned from these days that logic, when it is complex and/or when it changes decently often, needs to be externalized.
Business owners do not want to go to IT and submit a change request. They want to be able to see the impact of a change before they actually commit to the change. They want the agility to tweak their thresholds or rate tables within minutes or hours, not days and weeks. While there is testing needed for rules like for any software, it is much more straight-forward as it does not impact the code. It is just about QA testing and business testing.
Here is my answer:
I have been wondering the same thing. Several decades ago, I discovered expert systems at school, in my AI class. I fell in love with them, and even more so with rules engines as they were emerging as commercial products.
While coding is more powerful and intuitive than it used to be, the need is still there to make applications more agile. Having software developers change code is certainly more painful than changing business logic in a separate component, ie the decision service.
Some argue that the technology is difficult:
- syntax
- ability to find issues
Is the technology too difficult to use?
Because of syntax
I can attest that writing LISP back in the days was nothing ‘intuitive’. Since then, thanks God, rules syntax has improved, as programming syntax did too. Most business analysts I have worked with have found the syntax decently understandable (except for the rare rules engine that still use remnants of OPS5). With a little practice, the syntax is easily mastered.
Furthermore, advances in rules management have empowered rules writers with additional graphical representations like decision tables, trees, and graph. At Sparkling Logic, we went a step further to display the rules as an overlay on top of transaction data. This is as intuitive as it gets in my opinion.
Because of debugging
The second point seems more realistic. When rules execute, they do not follow a traditional programmatic sequence. The rules that apply simply fire. Without tooling, you might have to have faith that the result will be correct. Or rely on a lot of test case data! Once again, technology has progressed and tooling is now available to indicate which rules fired for a given transaction, what path was taken (in the form of an execution trace), etc. For the savvy business analyst, understanding why rules did or did not execute has become a simplistic puzzle game… You just have to follow the crumbs.
So why are rules not as prevalent as they should be?
Is the technology too easy to not use?
Because of ownership
I am afraid to say that IT might be responsible. While it is now a no-brainer to delegate to a database for storage, and for some other commonly accepted components in the architecture for specialized functions, it remains a dilemma for developers to let business analysts take ownership. You need to keep in mind that business rules are typically in charge of the core of the system: the decisions. If bad decisions are made, or if no decision can be made, some heads will roll.
Because of budget
Even if the management of decisions is somewhat painful, and inflexible, it is a common choice to keep these cherished rules inside the code, or externalized in a database or configuration file. The fact that developers have multiple options to encode these rules is certainly not helping. First, they can see it as their moment of fun, for those that love to create their own framework (and there are plenty of them). Second, it does not create urgency for management to allocate a budget. It ends up being a build vs. buy decision.
Without industry analysts focused exclusively on decision management, less coverage by publications, and less advertisement by the tech giants, the evangelization of the technology is certainly slower than it deserves to be.
Yet, it should be used more…
Because of Decision Analytics
I would stress that the technology deserves to be known, and used. Besides agility and flexibility (key benefits of the technology), there is more that companies could benefit from. In particular, decision analytics are on top of my list. Writing and executing rules is clearly an important aspect. But I believe that measuring the business performance is also critical. With business rules, it is very easy to create a dashboard for your business metrics. You can estimate how good your rules are before you deploy them, and you can monitor how well they perform on a day-to-day basis.
Because of ease of integration
For architects, there are additional benefits too in terms of integration. You certainly do not want to rewrite rules for each system that needs to access them. Rules engines deployed as a component can be integrated with the online and batch system, and any changing architecture, without any rewrite, duplication, or any nightmare.
Final Takeaway
With that in mind, I hope that you will not let these roadblocks to rules engines stop you. There are plenty of reasons to consider decision management systems, or rules engines as they are often called. You will benefit greatly:
- Flexibility to change your decision logic as often and as quickly as desired
- Integration and deployment of predictive analytics
- Testing from a QA and business perspective
- Measure business performance in sand-box and in production
- and yet, it will integrate beautifully in your infrastructure
See my response on Quora here: https://www.quora.com/Why-do-rules-engines-fail-to-gain-mass-adoption/answer/Carole-Ann-Berlioz
Learn more about Decision Management and Sparkling Logic’s SMARTS™ Data-Powered Decision Manager