Intellifest 2012: James Owen: Rules Benchmarks, 25 years going on

on October 25, 2012
SMARTS™ Data-Powered Decision Management Platform

For those of you who don’t know him, James is an old hand at AI, the person who started RulesFest (which became Intellifest) and a fervent supporter of benchmarking for rules engines. Read his blog, for example here: http://javarules.blogspot.com/2009/03/rulebase-benchmarks-2009.html

He delivered a history-laden assessment of rules benchmarking. Interesting and entertaining.

Benchmarking was absolutely essential a few years ago when the concern in the industry about the scalability of rules engine based solutions still existed. These days the good behavior of these engines under load is relatively well established, and in my experience the benchmarking done has moved from pre-sales and theoretical to post-sales and based on the actual problem being solved.

Benchmarks have value, I agree with James on that, in particular when attempting to compare engine implementations. It’s important, however, to understand what the benchmarks are really measuring. An inference engine has many aspects to it, and the benchmarks tend to be micro-benchmarks that only exercise a few of those aspects. These may or not be exercised in the case of a particular application. As somebody gathering comparative (or not) performance information, you need to extremely aware of this.

An important aspect too is that an application benchmark will exercise more than just the engine itself. As many have found out, it is frequent that in fact most of the performance bottlenecks are in the object marshalling from and to the application. If your marshalling involves serialization and communication, and the objects are large, it’s likely that optimizing that will yield the largest benefits.

Learn more about Decision management and Sparkling Logic’s SMARTS™ Data-Powered Decision Manager

Search Posts by Category

ABOUT US

Sparkling Logic Inc. is a Silicon Valley-based company dedicated to helping organizations automate and optimize key decisions in daily business operations and customer interactions in a low-code, no-code environment. Our core product, SMARTS™ Data-Powered Decision Manager, is an all-in-one decision management platform designed for business analysts to quickly automate and continuously optimize complex operational decisions. Learn more by requesting a live demo or free trial today.