Achieving High Decision Engine Performance at Scale
Decision engine performance, and, more broadly, the performance characterization of decisions is often a concern when considering decision engines. Specifically, how do these engines perform under high throughput, low response, high concurrency scenarios?
In fact, forward-chaining business rules engines (based on the Rete algorithm) developed largely to address some of these scalability challenges. Modern decision engines such as SMARTS™ go beyond Rete to optimize performance in different situations.
Understanding the End User
Before we dive into performance, let’s take a closer look at how organizations use decision engines like SMARTS™.
First of all, SMARTS™ can be utilized in a variety of modes:
- Interactive invocations: the caller sends data or documents to be decided on and waits for the decision results. For example, decisioning on online applications as they come in.
- Batch invocations: the caller streams large data or sets of documents through the decision engine to get the results. For example, decisioning on all customers after hours to prioritize call center calls for the next day.
- Simulation invocations: the caller streams large data and sets of documents through the decision to get the decision results and corresponding decision analytics computations. For example, running a Champion/Challenger experiment on transactions in the last year and generating reports to compare KPIs.
Second of all, SMARTS™ allows business users to implement decisions using a combination of decision logic representations, including:
- Decision flows
- Rules groups in rule sets
- Lookup models
- Predictive models
These different representations provide different perspectives of the decision logic. Users can choose the most optimal representation for implementing, reviewing, and improving the decision. For example, SMARTS™ decision flows visually organize a decision into high-level “workflows.” First, check the document data is valid, then apply knockout rules, etc.
Fine-Tuning Decision Engine Performance
In order to support these different logic representations in production, SMARTS™ executes them with dedicated engines tuned for high performance. For example, if you were to implement a predictive model on top of a rules engine, your result will typically be sub-par. However, in SMARTS™, each predictive model is executed by a dedicated and optimized execution engine.
Focusing on what is traditionally called business rules, SMARTS™ provides the following execution engines:
- A compiled sequential rules engine
- A compiled Rete-NT inference rules engine
- A fully-indexed lookup model engine
Let’s take a closer look at each type.
Compiled Sequential Rules Engine
This engine simply takes the rules in the rule set, orders them by explicit or implicit priority, and evaluates the rule conditions in the resulting order. Once a rule’s conditions has been met, it fires. If the rule set is exclusive, then the rule set evaluation is over. If not, the next rule in the ordered list is evaluated.
It’s actually more complicated, but that’s the gist. The important point is that there is no interpreter involved. Rules are executed in code compiled to the bytecode of the chosen architecture (Java or .NET or .NET Core). So, this executes at bytecode speed and gets optimized by the same JITs as any code in the same architecture.
A compiled sequential rules engine yields great performance when the number of transactions is very large, and the average number of rules evaluated (i.e. having their guards and premises evaluated) is not very large. For example, we’ve implemented batch fraud systems processing over 1 billion records for fraud in less than 45 minutes on 4 core laptops.
However, when the number of rules becomes very large (100K+ in a single rule set), then the cost of evaluating premises that do not match gets too high. In our experience, often this can be fixed by replacing your rules with a lookup model. Otherwise, a Rete-NT execution would be better.
Compiled Rete-NT Inference Rules Engine
This engine takes the rules within the rule set and converts them to a network following the approach described in this Rete algorithm blog post. Rete reverts the paradigm. Data is propagated through the network and ready-to-fire rules are added to the agenda. The highest priority rule in the agenda is retrieved and executed. The corresponding changes get propagated into the network, the agenda updated, etc., until there is nothing left in the agenda.
One important distinction with respect to the sequential engine is that in the case of the Rete-NT engine, a rule that already fired may well be put back in the agenda. This capability is sometimes required by the problem being solved. Again, there is much more to it, but this is the general idea.
SMARTS™ implements the Rete-NT algorithm, the latest optimized version of the algorithm by Charles Forgy, who serves on the Sparking Logic Advisory Board. Rete-NT has been benchmarked to be between 10 and 100 times faster than previous versions of the algorithm in inference engine tests. In addition, SMARTS™ compiles to the bytecode of the chosen architecture everything that is not purely the algorithm, allowing all expressions to be evaluated at bytecode speed.
As mentioned earlier, when your number of rules per rule set is very large (which can’t be simplified by a lookup model), the Rete-NT engine performs much better than the sequential engine. Our clients have used SMARTS™ with 200k+ rules in rule sets. These rules end up exercising hundreds of fields in your object model, and the Rete-NT advantages make these rule sets perform better than the pure sequential engine.
Fully-Indexed Lookup Model Engine
When rules can be summarized in a spreadsheet, leveraging a lookup model executed in a fully-indexed lookup model engine is ideal. For example, one shouldn’t have to recode an existing list of medication codes and their corresponding parametrized conditions into hundreds of IF-THEN statements. While in many systems, the search or lookup occurs outside a decision engine, there may be cases where you want it to be part of the decision. When that is the case, SMARTS™ allows users to upload the list in spreadsheet form and to create a Lookup Model with its own engine to specify how to query the data.
At a simplified level, in production, this engine works by essentially doing what database engines do: indexing the data as much as possible and converting the query into a search through indexes. Because the search is fully-indexed, scalability is superior to both the Sequential and Rete-NT engines.
Further Enhancements to Decision Engine Performance
SMARTS™ can also run concurrent branches of the decision logic, implemented in any combination of the engines above. This allows a single invocation to the decision engine to execute on more than 1 core of your system. There are heavy computation cases in which such an approach yields performance benefits.
Of course, performance is also impacted by deployment architecture choices, which I explore in my next post.
Learn more about Decision Management and Sparkling Logic’s SMARTS™ Data-Powered Decision Manager

