Call to Action

Webinar: Take a tour of Sparkling Logic's SMARTS Decision Manager Register Now

Rules Fest Live: Hal Hildebrand / Building a Distributed Rules Engine


Written by: Carlos SerranoPublished on: Oct 12, 20101 comment
Hal Hildebrand

Hal was lucky enough to be able to do research work at Oracle, even before the Sun acquisition.
His talk was about how he approached the architecting and implementing a Jess-based distributed rules engine, supporting a large-scale system (called C0) that also relied on Coherence. Hal’s long experience with OODBs and app servers certainly came through.

Hal made a number of interesting points.

Hal needed to make sure that not too many resources are used to manage (in particular not more than what is getting managed…), and that the decision-making is located at the natural place to avoid introducing more cost and coupling within the system.
One of the key decisions was the separation between local decision-making and global decision-making – and ensuring their proper separation of concerns and coordination.

Given the complexity of the distributed system and the event nature of its dynamics,  the system had to remain asynchronous and event-driven.

The rules engine was set up as the driver of the application, as opposed to a “consultant”.  With that architecture choice, the rules engine ended up in a mode where it continuously evaluated the observed state, applying the declarative rules to the state transitions.

The rules architecture was set up in a hierarchy, with goals becoming more abstract and declarative as you climb the hierarchy. The organization allows for breaking down the processing into local problems – where you have most control and information to do so – enabling robust parallelization. Global problems, on the other hand, where handled within a global context. This approach not only allows for efficient robust parallel execution, it also allows for minimal communication and friction.
Given that all is communicated through facts, the execution ends up being in-database fact manipulation with some Java execution.

Hal makes the point that while declarative systems (enabled by rules-based approaches) do not perform any worse than procedural systems, they exhibit much better representation and management characteristics. Amen!

In terms of the implementation, Hal described the work that needed to go into integrating Jess and Coherence. In essence the work consisted in having a Coherence continuous query on the cache to trigger the engine. All the state remained stored in the cache.
He explored the use of backward chaining to select which facts to consider – ie modifying the continuous query on the cache – this is pretty cool.
The approach actually went further, leveraging rules for the configuration, the workflow, etc… Declarative rules-driven everything!

Hal did need to make 1 key modification to Jess – essentially adding a GUID to uniquely identify facts in the cache.

Some metrics corresponding to the tests made:
– EC based cloud deployment
– 500 VMs with 10 system VMs
– 5000 managed processes
– 15% CPU load on active system processes, 2% network bandwidth (very good), and 1% of host CPU for managed nodes.
The system has not yet gone into production.

Hal walked us through a few future considerations for this system. This include (not restricted to):
– Implementation of ECA state diagrams into rules
– And… dynamic provisioning of rules set (debugging, analysis, monitoring) – which would be a very interesting development.

More futuristic, although interesting: taking this same model and apply to BPEL/SOA.

Call to Action

SMARTS Decision ManagerSparkling Logic SMARTS is a decision management platform that empowers business analysts to define decisions using business rules and predictive models and deploy those decisions into an operational environment. SMARTS includes dashboard reporting that allows organizations to measure the quality of decisions both during development and post deployment. Learn more about how SMARTS can help your organization improve decisions.
© 2022 SparklingLogic. All Rights Reserved.