Financial Services
How to implement a Rating Engine
Rating engine, pricing engine, compensation calculation, fee calculation, claims calculation, and many more types of projects aim at calculating a fee or cost. They typically share some complexity on the fine prints.
Implementing a rating engine has relied on different technologies over the years. The debate continues: can business rules be used for a rating engine? My answer is yes and no. I strongly believe that we need the full power of a decision management system. Let me explain.
When the rating engine logic is pretty straight-forward, business rules will do a good job. In the pay-as-you-go demo, we could summarize the pricing strategy in a few business rules. In real-life projects, compensation calculations and benefit calculations have also been implemented in a modest number of rules. As a rule of thumb (no pun intended), I would say that if you can express your rating logic as a formula, with or without a few exceptions here and there, you are in a good shape using business rules only.
Business rule engine is not enough
The limitation, though, is when these rate tables grow significantly. Some of these spreadsheets include tens of thousands of rows! This combinatorial explosion makes sense when you think of having rates specific to:
- 50 states,
- maybe up to 42,000 zip codes,
- 2 or 3 genders (some states can’t legally discriminate over gender, so you may deal with a gender-neutral pricing in addition to male and female)
- several risk score bins,
- many, many, many age options…
Writing all of these rules by hand may be a significant task. Rule import can help, of course, but keep in mind that it is likely a one-time solution. Updating and maintaining these rates over time will become a painful task, less painful than coding of course, but still painful. When actuaries come up with an updated rate table, finding the right line items could be a tricky exercise. My experience tells me that rare business experts provide a color-coded spreadsheet, highlighting only the changes. You certainly do not want to be in the business of comparing line by line the old and new rates.
However, the great advantage of business rules is their extreme flexibility. Assuming that the rating tables per se are taken care of, which I will cover on my next point, overrides for specific states, or product options, remains trivial by complementing the rating engine with business rules. Often, I see projects in which these overrides happen while or after retrieving rates. There is literally no end to the flexibility you can implement as business rules once rates are retrieved.
My first takeaway: I do like using business rules for fine-tuning rates, dealing with exceptions.
Rating Engines are mostly lookups
For rates that are dependent on this (sometimes) enormous number of lines to pick from, I prefer using lookup models. Lookup models are spreadsheets that can be used to return the matching columns. Let’s say you import a spreadsheet with state, age, gender, and score range. The lookup model will retrieve the rate that matches all 4 columns. You are not limited to one column to return though. In addition to the rate, the spreadsheet could return volume discount or any additional information.
The main advantage of using lookup models, in my opinion, is that there is no manual translation from spreadsheet to executable lookup model. The implementation effort is in the interface. Do you pass the actual gender for example, or do you translate its value to ‘neutral’ for those states that prevent you from using gender? Do you return a single rate, or is it possible to return multiple rates?
While rate tables can be as large as your business experts might dream, lookup models offer a great advantage in performance. They are indexed, and can return rates very quickly, somewhat independently of size.
My second takeaway: I do like using lookup models for retrieving rates.
Addressing evolving rates
As I mentioned before, the majority of the pain is typically in the maintenance of the rate tables in the rating engine. Using lookup models, this pain disappears. Updating rates is as easy as uploading a spreadsheet to the system. Overall, the 2 key aspects that I love about these mechanics deal with time-travel, seen from 2 different perspectives.
(1) The convenience of version tracking
Although rates change over time, you may need to resurrect these past rates. Many reasons come to mind. You might need to justify what your rates were at that time. Or, in urgency, you might need to backtrack bad rates that made their way into production.
Thanks to the underlying versioning system, and release management, these past rates are just a click away. Time-travel to the January 2020 release of your rating engine to resurrect pre-pandemic rates, or just to run simulations. You can also use the version history to promote that specific version of the spreadsheet to become current again. You will never lose any iteration of that spreadsheet that went into production. Peace of mind!
(2) The flexibility of time-sensitive rates
The introduction of new rates into your rating table is likely to follow a specific schedule. It happens that rates are just adjusted directly. But, more likely, they will start on a specific date, like January 1st. Rather than scheduling a job that will update the rates at midnight, it makes more sense to upload these rates upfront, and specify their effective dates.
Like business rules, rate tables can follow effective dates that apply to the entire sheet. Using the full power of business rules, you can activate these 2021 rates on January 1st for a group of states, and at a later date for another group of states.
I particularly appreciate that you have full control over the clock. Do you switch on January 1st at midnight at your headquarters? Should you adjust for the local time zone in each state? Do you consider invocation time? Or do you apply the proper rates based on delivery date? Once again, the sky the limit. I have yet to see a scenario that could not be implemented using a decision management system.
My third takeaway: I do like using lookup models for managing rates over time.
In conclusion, I highly recommend you consider a decision management system when implementing a rating engine. Business rules may have been limited in the past. However, decision management systems can certainly take you there now!
ETCIO: Equifax InterConnect helps Indian CIOs
ETCIO (an initiative of The Economic Times) has interviewed KM Nanaiah, country manager at Equifax. The article highlights the details of the tooling that is now available to financial institutions. They will see dramatic improvements in customer acquisition and loan decisioning.
InterConnect’s Rules Editor is our Sparkling Logic SMARTS decision manager. In particular, InterConnect customers have praised its Champion / Challenger capabilities.
A Year at Plug and Play
A little over year ago, Sparkling Logic joined the Fintech accelerator at Plug And Play. We were honored to be selected as one of 29 technology startups (from among 850 applicants) to join the inaugural launch of Plug and Play’s Fintech accelerator.
Plug and Play is the world’s largest global technology accelerator and venture fund with over 350 startups and 300 corporate partners. Plug and Play connects startups to corporations through vertical-specific accelerator programs. The goal of these accelerators is to help startups with funding, acquiring more customers, and fine tuning their business to better match customer demands. In our case, we joined the Fintech accelerator which was focused on financial technology and security. This accelerator was a good fit for us since many of our customers are in the Financial Services industry and are using our SMARTS Decision Manager product for applications such as fraud detection and risk management.
During our time at Plug and Play we have gained new customers and partners. We have also benefited from lots of exposure by presenting to over 80 companies. In meetings and discussions with these companies, we learned more about their needs and where our SMARTS decision management technology can help. This enabled us to refine our messaging and inspired the launch of a new pilot program to help customers develop a meaningful proof-of-value project using SMARTS.
Recently Plug and Play published a blog post and created a video featuring our CEO, Davorin Kuchan. In the blog post and video, Davorin shares more about our experiences during our year at Plug and Play. It will give you a sense of what life is like as a technology startup in Silicon Valley!
Are the Rules the Same for Everyone?
It might sound like a philosophical question, but there is a very practical aspect to it that we might not fully realize, when making decisions.
Business Rules execute consistently by nature. We like that. We want that.
There are exceptions though, when we would like to treat a segment of the population differently. We would like to always apply a conservative or aggressive strategy, but we might interpret that differently in different regions of the world, or for VIPs compared to occasional shoppers. Our decision logic is not so black & white after all.
While many organizations start with a simplistic design to address the variations between US and International, or between Gold customers and the rest, or between this and that… This approach does not stand for long. As soon as the number of variations grows, the duplication of business rules becomes a nightmare.
I recall a project that started with a thousand or two business rules. They decided to simply duplicate the rules for each customer configuration in their portfolio. Being very successful at what they do, they ended up with a lot of customer configurations. What do I mean by a lot? What would you say about 1.5M business rules? That was certainly more than the team could handle. As a result, the decision logic was behind, almost stale, as the business rules could not be modified promptly, as you would expect business rules to be. When changing a business rule, they would have to check each and every project to verify that the change was applicable to that customer configuration. A Hellish endeavor to say the least.
Equifax has the same challenge since they provide credit approval solutions to thousands of customers. They would certainly need an army of consultants if they had to duplicate their decision logic across their entire portfolio! In our presentation at BBC 2015, “Exceptions are no Exceptions”, Rex Keith (Equifax Senior Director, Product Management) and I will cover several perspectives related to the problem of exceptions, including this portfolio management challenge. We will also suggest design solutions to each challenge.
I hope that you will be able to join us in Vegas for this presentation.
If you miss us, or if you are looking for more information on this topic, I encourage you to watch our recorded webinar, Managing Complexity in Automated Decisions.