In the evolution from BRE to BRMS a decade ago, one new component made its debuts: the Repository.
James Owen had an interesting post on JavaRules blog on what a repository is and what benefits it bring (http://bit.ly/eOTwSh). I would like to elaborate on his take as a continuation to my BRE versus BRMS series.
What was there before?
In the old days, believe it or not, rules engines did not have any repository. I remember when ILOG Rules, the C++ ancestor of the JRules product-line, loaded a rules file in an embedded authoring UI (a plain old C++ class). It is hard to fathom that rule editing was happening in memory as part of the Production application. There was no formal separation between Development, Testing and Production as we consider now as the norm. To their credit, many customers were in the Telecom industry and used the product for Alarm Correlation and Filtering. Sets of fewer rules were created on the fly to decide how to deal with those many alerts. Having a multi-user environment to create rules “offline” was not perceived as a priority.
Blaze Advisor 2.0, the first release of our dear past product, was a little more visionary in the sense that it had a real authoring environment from the very beginning — actually they had carried on this vision in their original Neuron Data product called Nexpert Object — but it was still working out of files. The project files were basically persisted in the file system without any management capability on top of those files. As you can imagine, the team led by Carlos back then quickly addressed this shortcoming — in version 3.0 they introduced the repository. I remember that summer very well as it was also my debut at Blaze Software.
What is a Repository?
A rules repository is primarily a storage area for your business rules and other decisioning artifacts. You may want to compare it to a Database or Source Code Management System since it often relies on those technologies applied to Decision Management artifacts. As such it offers basic capabilities:
- Sharing: the main shortcoming, as I called it, was that files are okay for a single user working in isolation but when projects grow to several users then rudimentary sharing capabilities become unavoidable. Concurrent access causes the types of problems we addressed a long time ago with databases. Real-life modernization projects involve more than one owner of the business rules in general, especially as business users are empowered to create and experiment with this business logic. This significantly stresses the need for sharing.
- Storage of course: As I already elaborated in the previous post of that BRE versus BRMS series, storage capabilities offered by databases allow logic backup and other niceties.
- Versioning: As we have been used to in the past decades, tracking who did what and when is important, either for traceability or for convenient “roll-back”.
- Access Control: Although you may be happy to share the whole repository with your team, there may be value in partitioning the roles and responsibilities. At a minimum, repositories are configured with authentication. In many cases, you want to add authorization on top of it to grant privileges per product-line or geography. One scheme that is quite popular is to allow everyone to see the complete set of rules but to grant edit privileges to a smaller population e.g. American rules can be modified by the American team, European rules can be modified by the European team and Asian rules can be modified by the Asian team. In some places, we have seen maker-checker paradigms e.g. one person edits a rule and another person approves it.
Why is it so important for Decision Management?
All those basic capabilities drove the very early releases of rules repositories in the industry but the real value-add, in my view, comes from those higher-level capabilities:
- Search: When rules repositories grow, and they often grow significantly in the hands of business users, finding a rule may be as tricky as finding a needle in a haystack. Early search tools relied on rule name that you may not know as a business user. More powerful search capabilities focus on the content of rules: which rules look at age and/or marital status? The ultimate search capability, that is not widely available yet, allows you to search on the semantic content. Instead of looking at age (which you could parse at a string), you may only want to look at rules that impact the elderly — in which case in want to be able to retrieve any rule with conditions such as age > 80, age >70, age between 90 and 100, etc. but not age < 25. It may require to agree on the semantics since a rule that applies to anyone older than 30 does apply to older people too. How strict do you want to be?
- Verification / Validation: Proactively searching for what might be wrong is another very valuable service that became possible with rules repositories. Verification is a static analysis of your business rules to detect potential anomalies such as gaps, overlaps or any kind of inconsistencies. Validation is a brute-force service that actually executes business rules against predefined test cases and highlights discrepancies between the expected outcome and the actual result. Combining both allows Knowledge Workers or Subject Matter Experts (SMEs) to focus on potential problem before they are deployed. I am cautious here as a gap or overlap is not necessarily a bug in your logic. Take marketing offers: one purchase may qualify you for several discounts — a free magazine subscription AND free shipping.
- Structuring: The ability to add meta-information (aka tags) to business rules came a few years after the introduction of rules repositories. It is clearly very useful for finding business rules based on some criteria that may be independent of the content of the rule. For example, you may want to look for all changes made by Carlos in the past 3 months or you may want to look for business rules that have been finalized but not yet approved. I really like the idea of using tagging for structuring your repository. Folders are commonly used but they suffer some limitations. It happens frequently enough that one item can be stored in more than one place. Using tags for filtering or structuring allows you to view that one item in both places without duplicating it. You may be familiar with this concept in your gmail account. Very few BRMS products support that capability to this day as far as I know.
- Discovery / Catalog: At a higher level, one of the most important capabilities offered by the rules repository is the ability to catalog your business logic. When your rules are stored in a repository, you can reuse those entities over and over and find out where you have been using them. Take an Audit scenario for example, you may need to find all the projects that are actually leveraging a given score model. We have also seen the opposite scenario where a score model performance is deteriorating and you want to refresh it. Although you certainly have your scoring formula in your source code management system or in your rules code, tracing back to the original Analytics project may be more problematic. Back in the early 2000’s, the Rules Traceability Initiative attempted to address that use case for requirements. Meta-Information could be leveraged again for traceability.
- Lifecycle Management: My definition of Lifecycle Management is the service that enforces a pre-defined promotion process. It is up to you of course to decide how rigorous this process is. It is common to automate the execution of verification and validation services. A supervisor may have those reports delivered to his/her queue before approving manually the set of rules to be promoted. The set-up of a formal process for Lifecycle Management adds transparency to the business changes made into the system. You can store the flux of rules for identifying which rules were in effect at a given point in time and possibly roll-back to a particular state.
The rules repository has been a huge progress in the BRMS evolution because it enabled all those value-added services and more. I am excited to now see an expansion of that concept to Decision Management such that rules and other decision artifacts such as score models are finally making their way into the repository.
Learn more about Decision Management and Sparkling Logic’s SMARTS™ Data-Powered Decision Manager