IT
Regression Testing and Business Verification
How to Ensure Your Modernization Project is a Success Part 3
In our previous post, we suggested that you establish a team early on to mine your database for edge cases and other examples that can be used for regression testing, define expectations ahead of time based on those examples, and test-as-you-go. In our final post of this series, we’ll go deeper into testing.
Why is Regression Testing So Important?
Regression testing is done to validate that an application functions as expected after a code change or other update. In decision migration projects, regressions tests ensure that the decision outcomes in your new system match your expectations. What’s important to remember is that you’re comparing decision outcomes and not the rules themselves.
Another reason that regression testing is a must-do is that it will help you identify what we call “logic band-aids.” These are those messy workarounds that you had to do outside your legacy system to make it work properly. Usually these workarounds exist because your legacy system lacked enough expressivity and/or functionality on its own. Logic band-aids can live in various places, including your code, your database, or your UI. Finding them is one of the hardest parts of the migration project. Regression tests can clue you in on where to look. And once you find them, you should replace them with easy-to-understand decision logic on the new system.
Regression Testing Best Practices
Regression testing can be time consuming, so here are ways that you can speed up the process (and your overall migration project):
-
Include regression testing capabilities as part of your new technology evaluation:
As regression testing should not only be a part of your migration process but also an ongoing part of decision management, your new system should allow you to run regression tests easily. What to look for include the ability to run hundreds of thousands (if not millions) of test cases quickly, the ability to easily filter out test cases where outcomes do not meet expectations (ex. add QA flags), and the ability to report on decision outcomes (ex. custom dashboards). This short video shows you how you create dashboard reports in SMARTS to monitor and compare KPIs when you modify decision logic.
-
Build your library of test cases as part of your planning process:
We already mentioned this in our previous post but want to reiterate it here. Ideally, by the time you start your migration, you should have your test cases ready, so that you can test-as-you-go. In addition, once you have your library, you can continue to use these test cases for regression testing every time you modify your decision logic moving forward.
-
Augment regression testing with business verification:
Regression testing only works when your old system is error-free. So even if your decision outcomes in the new system match what’s in the old, the outcomes may not be accurate. This is where business verification comes in handy. These are additional business rules applied to your business rules to validate that your decision logic is correct. For example, you may have a rating table that segments people into different tiers where the higher the tier, the higher the rating. You can create a verification rule that checks whether Tier 1 < Tier 2 < Tier 3, etc. If Tier 3 < Tier 2, you know you have an error in your table. Focus your verification rules on higher-level patterns like this to help you catch any errors from your old system. And like with regression testing, continue to run these checks every time you modify your decision logic post migration.
Want to learn how SMARTS empowers business analysts to make smarter, faster operational decisions? Contact us today to request a custom demo.
Decision Migration Techniques & Best Practices
How to Ensure Your Modernization Project is a Success Part 2
In our previous post, we discussed why migration should be a part of your decision modernization planning process. In today’s post, we’ll cover decision migration techniques and best practices. As a reminder, while this blog series provides tips for any modernization initiative, our focus is on projects related to operational decision-making and decision management.
Decision Migration Techniques
Decision migration projects are similar to other migration projects. In general, there are three techniques with variation in-between:
- Fully Automated
- Rule Conversion
- Manual Rewrite
While we believe a manual rewrite produces the best quality, this technique is also the most labor intensive. Therefore, blending the three techniques will probably be the best approach for your organization. Let’s take a closer look at each technique.
Fully Automated: automatically translate decision logic that is reliable and stable
- How it works: extract your rules (decision logic code) as data and then apply business logic to convert that data to the decision logic representation of your new system
- Pro and Cons: fastest technique but will transfer over any errors and ugly code from the old system — the decision logic in the new system may still be difficult for business analysts to understand and manage
- When to use it: where your decision logic is reliable (error-free) and stable (doesn’t change)
- What to look for when evaluating new technology: automated import capabilities
Rule Conversion: literally translate decision logic that business analysts are familiar with
- How it works: manually translate rules from the old syntax to the new syntax
- Pro and Cons: requires less skills/time than a manual rewrite and can incrementally improve the readability of the rules but prevents you from leveraging the richer features of the new system
- When to use it: pockets of decision logic that business analysts were already comfortable managing in the old system (ex. a decision table)
- What to look for when evaluating new technology: expressivity of the language
Manual Rewrite: completely rewrite ugly and outdated decision logic
- How it works: redesign the data model, simplify logic, and address decision limitations
- Pro and Cons: enables you to take advantage of the richer features of the new system to produce the best quality (redesign decisions and decision management for business analysts) but requires a significant upfront investment (skilled labor)
- When to use it: crucial areas of your decision logic that currently cannot be effectively managed by business analysts
- What to look for when evaluating new technology: verification capabilities
A way to blend manual rewrite with fully automated could be to focus the redesign on the decision flow (sequence of decision steps). Once the decision flow has been established, the rules that operate at each decision step can be fully automated from the old system. Regardless, determining your migration approach will take some time (and why migration should be a part of your decision modernization planning process).
Decision Migration Tips
To help you plan and speed up your migration approach, here are a few tips:
- Start with the data model: Whether you decide to keep your existing data model or create a new one, you should look for opportunities to augment. In addition, establish a team early on to mine your database for edge cases and other examples that can be used for regression testing.
- Get rid of ugly code: Where possible, remove concepts that are confusing such as null, nested logic, and string assembly. Instead, replace them with syntax (in place), business terms, calculations, and functions. In other words, the more expressive you can be, the better.
- Define expectations ahead of time and test-as-you-go: Does an outcome on the new system have to exactly match the outcome of the old system or can there be variability? Once you establish expectations, be sure to test as you build decision logic (makes it easier to identify errors than waiting til the end to QA), using the edge cases that you identified early on.
Through proper planning and utilizing the tips we mentioned, you can avoid dragging out your migration project for months, if not years. For example, one of our bank clients was able to migrate a multi-channel credit origination system onto our decision management platform in 2 weeks! But more importantly, because they understood the importance of testing, they were able to improve the quality of their decisions from the get-go. In our final post, we’ll go deeper into regression testing and business verification during migration and beyond.
Want to learn how SMARTS empowers business analysts to make smarter, faster operational decisions? Contact us today to request a custom demo.
Decision Migration & The “Golden Rule”
How to Ensure Your Modernization Project is a Success Part 1
When it comes to modernization, migration is often overlooked in the planning process. It’s no wonder that digital transformation failure rates have been upwards of 70% according to major consulting firms. Recently, Carole-Ann Berlioz, Co-Founder and Chief Product Officer at Sparkling Logic shared her insights from client migration projects in our webinar An End-User Approach to Modernizing Operational Decision-Making. In this 3-part series, we dive deeper into the webinar key takeaways:
- First, include migration support capabilities as part of your new technology evaluation
- Second, pre-plan the migration project so it doesn’t take years
- Finally, test, test, test!
While the blog series provides tips for any modernization initiative, we will focus on projects related to operational decision-making and decision management.
The Case For Decision Modernization
Operational decisions are the day-to-day, repetitive, and often high-frequency decisions that keep an organization running and impact an organization’s bottom line. Examples include:
- In Insurance, whether or not to accept an insurance claim
- In Financial Services, how to price a credit offer for an applicant
- In Retail, how to efficiently get your products to your retail locations
- In Healthcare, what treatment plan to recommend to a patient
In an effort to gain more operational efficiencies, organizations have invested in automating many of these operational decisions. However, more often than not, the focus was on processes and not on the decision-making itself. As a result, the code responsible for decision-making is typically buried within various systems. Consequently, managing decisions is cumbersome and requires IT intervention. Even small changes to decision logic would require going through the system development life cycle, preventing organizations from adapting quickly to market, regulatory, and other external changes. Therefore, many decision modernization projects involve isolating the code responsible for executing decision logic and treating it as a separate asset to enable the business to:
- Ensure the consistency of decisions
- Rapidly adjust decision logic
- Improve decision quality over time
Why Migration Should Be a Part of Your Decision Modernization Planning Process
From use-case specific technologies like loan origination software to universal decision management platforms like SMARTS, there are many technologies available to choose from. Regardless of whether you build or buy, migration should be front and center of your modernization planning process. That’s because how you migrate will determine how well your new system will serve the end user — the business analysts responsible for managing those operational decisions.
This brings us to what we call the “Golden Rule” of Decision Migration: think like a Business Analyst because the future of decisions lies in their hands. And in our experience, the #1 thing business analysts want to avoid is code. In our next post, we will cover decision migration best practices to minimize code.
Want to learn how SMARTS empowers business analysts to make smarter, faster operational decisions? Contact us today to request a custom demo.
Data vs. knowledge in automated decision management — Why not both?
In the tech industry, we also have our well-known “Coke vs. Pepsi”, “Avis vs. Hertz”, or “Mac vs. PC” debates. In the automated decision management category, the question that keeps coming up is “data vs. knowledge.” The aim of this blog post is to show that from a practical point of view, data and knowledge can be found in the same application. To do this, we will show it with SMARTS, Sparkling Logic’s decision management platform that allows users to combine data and knowledge without them entering the “data vs. knowledge” debate.
Origin of the debate
The “data vs. knowledge” debate dates to an old debate about whether knowledge about a subject should be hand coded or machine learned. A first camp of researchers and practitioners sought to encode this knowledge in the form of rules and an inference engine that runs on these rules to supply answers to user questions. A second camp sought to develop programs that learn from available data using statistical methods to generate models that can make predictions from unseen data. At Sparkling Logic, we support a pragmatic approach that consists in using data, knowledge, or both depending on the problem and the situation at hand.It is all about the situation
There is no such thing as a stand-alone decision management application. It is often built with the purpose of being integrated into a larger system for loan origination, risk management, product configuration, or other similar applications. As I wrote before, there is no one single approach. It is all about the situation.Data is everywhere, easy to collect, organize, and transform into predictive knowledge. So, if you have a lot of data, it may be better to build your decision management application around that data if the new observed data does not deviate too much from the old, learned data.
On the other hand, when you have knowledge whether in the form of rules or procedures, it is better to build your application around this valuable knowledge if it is easy to capture and code into the application.
If you have both data and knowledge, why not using the two, when you can do so in a modern decision management platform such as SMARTS, the subject of the next section.
The SMARTS way
SMARTS is a decision management platform that enables creating, testing, deploying, and improving automated decisions in an integrated platform. I will not detail it here, but you can find a brief overview of SMARTS on our blog page and a full description on our resources page. Instead, I will focus the rest of this article on how to use SMARTS when you have plenty of data or domain knowledge about the application you want to develop.You have plenty of data
For situations where you have plenty of data, SMARTS proposes two tools: RedPen and BluePen.
With RedPen, you write decisions in the form of rules using a use-case driven approach. A loaded data sample supplies the context for the rules and enables immediate execution and testing of each rule. RedPen mimics what subject-matter experts do when they flag decisions.
When you activate RedPen, you can pin an existing rule, a field of this rule, or a rule set and change it as if you were using a real pen on real paper. You can also create new rules with RedPen, SMARTS automatically turns them into executable rules.
On the other hand, BluePen lets you explore and analyze your data using your domain knowledge to find predictors. Then, using these predictors, you can generate a model in the form of legible rules and integrate them into your decision logic.
Using BluePen, you can engineer or change the models any time you need to. Without heavy investment in data analytics tools and efforts, you can evaluate BluePen models in simulations and quickly deploy them in the context of an operational decision.
You have domain knowledge
For situations where you have knowledge, SMARTS proposes two additional tools: SparkL and Pencil.
SparkL is Sparkling Logic’s language for writing rules in a natural language fashion. SparkL comes with everything you need to write rules —mathematical expressions, string manipulations, regular expressions, patterns, dates, logical manipulations, constraints, and much more. You can express any imaginable decision logic and symbolic computation, making it the choice for highly sophisticated decisioning applications where the conditions as well as the actions can take a wide variety of forms.
Pencil is our DMN compliant graphical decision design tool for uncovering, documenting, and sharing decisions with colleagues and partners. With Pencil, you just drag and drop graphical shapes to form a complete decision diagram. Then you add business logic to the graphical shapes and let SMARTS execute it.
Pencil helps you think about the ultimate decisions in a structured way, starting from the top-level decision to smaller sub-decisions. This iterative process is very friendly and amazingly easy to share with colleagues or partners working on the same project.
In addition to the above tools and language, SMARTS comes with a built-in dashboard to measure and improve business outcomes of the taken decisions.
You have both
If you are lucky enough to have both data and knowledge, you can leverage your models’ outputs by using it as the input to rules. For example, your loan management application could run a model that calculates a score and another model that calculates a risk and use that score and risk in a rule to calculate a price.
You can also do it the other way around, using the outputs of rules as inputs to your models that you would have trained with data. For example, your application might run a rule to classify a loan applicant, then run a model to calculate their risk of default and another model to calculate the price.
Whether you have data, knowledge, or both, SMARTS uses them as sources of information for the automation of your operational decisions.
Summary
- Data and knowledge do not have to be antagonistic. They can both be used as inputs to automate decisions.
- SMARTS is a modern decision management platform that enables their combination in an elegant and seamless way. For SMARTS, data and knowledge can be used as sources of information.
- When you have a lot of data, you can use RedPen to write rules without learning a special rule language or syntax, just starting with the data. You can also use BluePen to learn from data and turn it into rules.
- When you have knowledge, you can use SparkL to encode it into rules, from the simplest to the most complex rules that your application may require. You can also use Pencil when designing, documenting, and sharing your decisions are part of the requirements.
- Our mission is to enable customers to implement the most demanding decisioning requirements and to easily change and improve them over time. Whether you have data, knowledge, or both, we can help. Just contact us or request a free trial.
About
Sparkling Logic is a company at the forefront of technological innovation in decision management. We help businesses automate their operational decisions with a powerful decision management platform, designed for business analysts first.Sparkling Logic SMARTSTM (SMARTS for short) is a decision management platform that enables creating, testing, deploying, and improving automated data-based decisions in an integrated easy-to-use environment.
Hassan Lâasri is a data strategy consultant, now leading marketing at Sparkling Logic. You can reach him out at hlaasri@sparklinglogic.com.
SMARTS as regulatory compliance technology
So far, we’ve covered how to use SMARTS for decision management, micro-calculation, and data transformation. In this blog post, we show how you can use it to implement regulatory technology (regtech).
Regtech
Regulations, from Basel rules on bank capital requirement to Sarbanes-Oxley Act on corporate financial statements, to MiFID on pre- and post-trade transparency requirements across EU financial markets, have forced regulated companies to develop processes to find, assess and mitigate risks. To comply, investment firms, retail bankers, and insurance companies have turned to regtech for help.Regtech is an acronym for governance, risk, and compliance management technologies in companies, more particularly those working in highly regulated industries such as financial services, insurance, and healthcare. They allow the implementation of legal requirements in the nervous system of companies, in front-, middle- and/or back-office.
All regulations being prescriptive, it was natural that rule-based systems were among the first technologies used, with varying degrees of success which we will detail here, before showing how SMARTS overcame them.
SMARTS as regulatory compliance technology
Your data is uploaded and transformed in the same tool
Highly regulated companies must deal with continuous growth in transaction volumes and a massive accumulation of data that they must ingest or produce daily while constantly complying with ongoing regulations. They could do it, but they had to use other tools besides rules-based systems and they either had to connect them or transfer the data back and forth. With SMARTS, they don’t have to, since it allows data to be uploaded and transformed into rules or calculations in the same tool.
Your data is turned into insights and your insights into decisions
Before the widespread use of big data and analytics, investment bankers relied on complex analysis of information using statistical learning. Today, the entire financial services and insurance industry has integrated advanced data analytics into its artillery to detect market signals and predict market trends. The most advanced want not only to transform data into insights, but these insights into decisions. SMARTS was one of the only tools if not the only tool to offer an integrated solution to transform such companies into lifelong learning organizations where data helps find opportunities and risks, machine learning turns that data into knowledge and rules transforms this knowledge into decisions, thereby closing the virtuous circle that data promises.
You limit your risk for noise, errors, and biases
Since the advent of data, regulators have been closely monitoring the bias issues of automated decision-making systems, particularly those that rely solely on data and use machine learning to calculate scores and then decide instead of a human. In SMARTS, users implement decisions in the form of business rules, decision trees, decision tables, decision flows, and lookup models. All these intuitive representations make decisioning self-explainable so that they can test decisions individually as well as collectively. So, at any time, they can check potential noise and errors before they translate into biases.
Your data and transactions are tracked in real-time
A key need of the financial services industry is real-time responsiveness to suspicious events such as unusual transactions that may show fraud, money laundering, insider trading, or may not be unusual in themselves but nevertheless exceptional in relation to other transactions before or after. Based on their experience in the earlier generation of decision management systems, the founders of Sparkling Logic decided to integrate real-time decision analysis from the ideation of SMARTS. The product has always integrated a dashboard that tracks data and transactions so that the user can react by changing rules in real time.
You react very quickly before an error, an anomaly, or a fraud spreads
Companies must make thousands of complex risky decisions – monetary, reputational, and legal risks. For example, in every decision they make, there are tiered combinations of terms and conditions, legal constraints, eligibility criteria, and levels of risk involved. Rule-based systems allowed them to implement these decisions in the form of tables or decision trees, or rules, but at the expense of side effects on the business. With SMARTS, they graphically define KPIs and drag and drop them into a dashboard to visually check the impact of each decision or group of decisions on business performance. Users can also set thresholds and define patterns which if reached will trigger notifications and alerts. This way, the users will be able to react very quickly before an error, an anomaly, or a fraud spreads and results in enormous damage.
Your system is easy to maintain and upgrade
Prior to the emergence of regtech as a hot technology, highly regulated companies used rules engines to encode the directive logic of laws, regulations, and internal policies. Additionally, they could implement complicated decisions with tens of thousands or more if-then rules. All went well until they discovered that, like any hard-coded software, rule-based systems could be complex to maintain. With SMARTS, they don’t code and hope the code is correct. They create, test, deploy, run, monitor, and change graphically through web forms and point-and-click. Therefore, systems developed with SMARTS are easy to maintain and upgrade.
Wrap-up
The SMARTS is not strictly speaking regtech in the sense that it does not come with all the code of financial and insurance regulations, but it allows them to be implemented quickly and explicitly in the form of rules, trees or graphs. This makes the code easier to change if regulations change as they often do, such as Basel which is in its third version and MiFID in its second version.SMARTS not only eases the implementation of governance, risk and compliance rules, but it also facilitates their monitoring in real-time. SMARTS not only eases the implementation of governance, risk and compliance rules, but it also facilitates their monitoring. KPIs, dashboards and metrics were fundamental from the start of the product and not an afterthought once the product was released.
If you envision modernizing the implementation of a regulation, be it Basel, Sarbanes-Oxley, MiFID, GDPR, or any other regulation, SMARTS can help. Just contact us or request a free trial.
About
Sparkling Logic is a company at the forefront of technological innovation in decision management. We help businesses automate their operational decisions with a powerful decision management platform, designed for business analysts first.Our motto is “your decisions, our business.” Using SMARTS, organizations have automated complex decisions in days, not weeks, or months. Our mission is to enable customers to implement the most demanding decisioning requirements and to easily maintain and improve them over time.
Sparkling Logic SMARTSTM (SMARTS for short) is a decision management platform that enables creating, testing, deploying, and improving automated data-based decisions in an integrated easy-to-use environment.
Unlike other tools that focus solely on the authoring and maintenance of business rules, SMARTS is decision-centric and focuses on measuring and improving business outcomes in the context in which our clients work, especially with complex regulations. Major enterprise customers like Equifax, First American, SwissRE, Centene, and NICE Actimize integrate SMARTS in their core systems.
Our customers — who they are, what they want, and what we bring them
As an enterprise IT solution, SMARTS has different customers in the same organization who directly use it or indirectly benefit from it. This blog post aims to succinctly describe who our customers are, what they want, and what value we bring to each of them that matches their unique needs.
Business analysts
In our terminology, these are the customers in the organizations who design, author, deploy, and update decisions according to the company’s policies and industry’s directives.
When they looked for a decision management solution, they looked for product simplicity and rich functionality. More importantly, they wanted autonomy once the solution was in place.
After a few weeks after training on SMARTS, our business analyst customers reported that they very much liked to have data, models, and business rules in the same tool. They enjoyed how we succeeded in managing SMARTS’ evolution to have both richness and easiness in the same product. They also enjoyed being able to quickly author, test, deploy, run, monitor, and change decisions. Their experience with SMARTS was a joy as they could focus on the decisioning process and its outcomes instead of the technology to implement it.
Business users
Business users are the people who run, monitor, and manage the performance of the business. In our case, they are the internal customers of business analysts. They are the ones who use the solution daily.
They wanted to know how easy it will be for them to monitor decisions built by business analysts and make the necessary changes when the actual performance may deviate from the expected performance.
After using SMARTS, business users reported the following benefits: Quick change-test-deploy-run cycles, being able to work without coding and with no prior knowledge of machine learning or business rules, just with their knowledge of the business and using web forms and point-and-click.
IT
By IT, we designate IT the people who install and connect the solution to the rest of the organization’s IT system. They asked for integration, performance, security, and fit with the IT global architecture and governance.
They want to have business analysts and business users to be autonomous but at the same time being able to monitor the solution as the rest of the IT infrastructure.
IT people liked all the performance, security, integration, and scalability we promised. They also appreciated SMARTS adherence to the enterprise IT architecture and governance as expected. They liked how easily they could deploy SMARTS on premises or in the cloud. Finally, they also very much liked to have no additional development or changes in the current applications.
Data scientists
These are the people who develop and manage models using data science libraries through languages such as Python, R, SAS, and SPSS.
They are not direct users, but they were willing to see their models fully operational into the new solution while they continue their effort on enhancing existing models and experimenting with new ones.
Thanks to SMARTS, they were able to know the performance of their models in production with real data and transactions. SMARTS was an effective demonstrator of their models.
Management
In our case, these are the people who head organizations or verticals where decisions are at the core of their operations, throughout all the organizations activity. Their attention is “more revenue, less cost, and why not both!”
They wanted to hear about similar successful implementations in their market, in particular the time it would take to recoup their investment in the new solution, and the strategic advantages it will provide them after one year or two in production.
To management people, we brought strategic benefits. They could operate the business under a decisioning process that implements the business strategy. Their organization could finally make informed, error-free, and unbiased decisions. And they were insured that the decisions taken were in full compliance to internal policies and industry regulations.
About
Sparkling Logic is a Silicon Valley company dedicated to helping businesses automate and improve the quality of their operational decisions with a powerful decision management platform, accessible to business analysts and ‘citizen developers.’ Sparkling Logic’s customers include global leaders in financial services, insurance, healthcare, retail, utility, and IoT.
Sparkling Logic SMARTSTM (SMARTS for short) is an all-in-one low-code platform for data-driven decision-making. It unifies authoring, testing, deployment, and maintenance of operational decisions. SMARTS combines business rules with predictive models to create intelligent decisioning systems.
If you envision modernizing or building a credit origination system, an insurance underwriting application, a rating engine, a product configurator, a condition-based maintenance application, or such applications, SMARTS can help. Just contact us or request a free trial.
Decision Management Systems: What Are They and When to Use Them?
The purpose of this blog post is to succinctly introduce decision management systems for those who are new to the field. First, I present what decision management is, then the technologies that make it up. I also present where they are used and what they bring to the companies that use them.
What is decision management?
There are two types of decision-making technologies. The first are descriptive in that they implement how people make choices among alternatives based on their beliefs and preferences. Think of a doctor deciding a treatment following a diagnosis or a trader buying an asset following a predictive model. The second are normative in that they implement regulations, policies, or strategies regardless of the beliefs or preferences of those who follow the decisions. Think of a loan officer deciding based on an applicant’s repayment history or an insurer calculating the premium an applicant should pay based on the applicant’s medical condition.There is no single definition that differentiates the two technologies. You have heard of knowledge-based systems, expert systems or reinforcement learning for the former technologies and decision tables, decision trees or business rules for the latter. But there is a consensus to name the second type of technologies decision management systems. So, when you hear or read someone referring to decision management, think of prescriptive methods, technologies, products, and systems implementing formal laws, industry regulations, company policies, and business strategies.
What are the underlying technologies?
Often decision management systems are confused with business rule engines, but they are more than that. Behind the terminology of decision management systems hide multiple technologies. The simplest are decision tables, trees, and graphs. The most sophisticated combine rules and predictive models. If we take the example of SMARTS, it integrates eight decision engines into the same platform. Depending on the problem at hand, one may choose one or the other, or even combine them in the same set-up.Also, the users of modern decision management systems are not IT people anymore but businesspeople instead. So, they come with features that allow non-specialists to use them without IT intervention. With modern decision management systems, IT only takes care of the first installations and configurations, the systems come with everything necessary to ensure the governance and security of the applications developed as well as the ease of integrating them into the corporate IT architecture. If we take the example of SMARTS again, it comes with an easy-to-use graphical authoring interface, pre-deployment rule testing, rule repository with version control and rollback, large-scale simulation, real-time decision performance monitoring, and much more.
Where decision management systems are most used?
Like any technology, decision management systems are not a one-size-fits-all solution for every decision problem. They are not suitable for long-term or midterm slow decisions that companies make once a year or a quarter. In these cases, optimization technologies are more used. They are not also suitable for cases with uncertainty and where probabilistic technologies such as probabilistic graphical models are more used.Decision management systems are best suited when there is a substantial number of decisions and calculations that are often nested, often invoked, and likely to change often. Therefore, one must consider decision management systems for the operational and day-to-day decisions that companies make in the thousands and sometimes millions in a single day. We find these cases in banking for credit risk assessment, in insurance for premium calculation and even in retail for product configuration.
Although not dedicated to finance, insurance and healthcare, customers of SMARTS have widely used it for loan origination, risk management, fraud detection and money laundering prevention. These are typically cases where organizations make decisions and calculations thousands and sometimes million times a day and may change based on the market dynamics or global economy, or updates to regulations or business strategy.
What are the key benefits of using decision management systems?
Decision management systems come with two critical benefits. As decisions are explicit, they make it possible to understand and explain the decisions implemented so that one can change them more easily when a new situation requires it. They also reduce the risk of errors and biases met in customer-facing applications, such as recommendation, credit, or insurance systems based solely on machine learning.Key takeaways
- There are two sets of decision-making technologies. The first is descriptive. The second is prescriptive. Decision management systems are descriptive in that they implement formal laws, industry regulations, company policies, or business strategies.
- Decision management systems are best suited when there are thousands or millions of decisions and calculations that are often nested, often invoked, and likely to change often.
- The power of modern decision management systems lies less in the engines but more in the features surrounding them to enable businesspeople take full control of the decisioning process without heavy IT intervention beyond first installation and default configuration.
- Decision management systems have two key benefits. They ease the integration of changes in regulations, policies, and strategies. They also reduce errors and biases in customer-facing applications.
Where to look for further information?
We write regularly about decision management systems and SMARTS. You can find quite a bit of information on our blog and webinars. You can also download our white paper, request a demo of SMARTS, or try it.About
Sparkling Logic is a Silicon Valley company dedicated to helping businesses automate and improve the quality of their operational decisions with a powerful decision management platform, accessible to business analysts and ‘citizen developers.’ Sparkling Logic’s customers include global leaders in financial services, insurance, healthcare, retail, utility, and IoT.Sparkling Logic SMARTSTM (SMARTS for short) is an all-in-one low-code platform for data-driven decision-making. It unifies authoring, testing, deployment, and maintenance of operational decisions. SMARTS combines business rules with predictive models to create intelligent decisioning systems.
Hassan Lâasri is a data consultant and interim executive, now leading marketing for Sparkling Logic. You can reach him at hlaasri@sparklinglogic.com.
SMARTS for data marketplaces
In my earlier blog post, I explained how decision management and business rules were suitable for micro-calculations, the type of computations that businesses often codify into large spreadsheets and use to score, rate, or price items. In this post, I explain how they are also suitable to simplify data integration, aggregation, and enrichment when building and running data marketplaces.
Data marketplaces
Data marketplaces are a shift from data warehouses where the goal is not only to store large volumes of data, but to make that data be consumed as a service without resorting to IT or prior knowledge of a query language. Data marketplaces are often organized into three layers:- At the lowest layer, we find raw data stored in the form in which it was ingested from the data sources. Data sources can be global ERP and CRM systems, or even local MySQL databases and shared Excel spreadsheets.
- At the middle layer, we find integrated data from multiple sources that is reconciled to resolve disparities and inconsistencies found in the original data. Often, the source systems do not have the same format for dates, names, phone numbers, and addresses. Sometimes the same object can have different attributes in different data sources.
- At the topmost layer, we find aggregate data expressed in summarized forms, often to inform about groups rather than individuals. It is at this level that we also find data enriched by external data to make them directly consumable by the businesspeople.
To learn more about data marketplaces, I recommend the Eckerson Group white paper: The Rise of the Data Marketplace – Data as a Service by Dave Wells.
Easy to define, hard to construct
Defining a data marketplace as we have just done is simple, its construction is complicated for two reasons:1) Data heterogeneity. It is not enough to bring together all the company’s data in a data marketplace for the data to be transformed into knowledge, forecasts, and decisions. Indeed, all data does not have the same age, the same structure, the same format, the same quantity, the same quality, and above all the same utility. If an attribute is important for a business line, it is not automatically important for another business line, yet within the same company.
Each business line has its vision of the product, of the customer, and of any entity managed by the various actors of the company. In the luxury sector for example, a dress, a bag, or a piece of jewelry, although it is a unique object, is seen through different attributes according to the databases where this same object is stored. Looking to exploit all the data available in a company to extract predictions and then decisions not only require integration but also transformation, unification, harmonization, and enrichment.
2) Data reorganization. A data marketplace would work better if data, information, and needs were always stable. But in a dynamic and rapidly changing business world, groups are reorganizing, and companies are acquired, merged or separated. For example, to simplify finance reporting, a country can change the region it was in a year ago, and it’s a safe bet that it will change yet again if a new boss is appointed, or a region is split or added. To be successful, data marketplaces must be implemented as change-tolerant projects.
These two reasons are representative of situations where decision management technologies are used: piecing together things that move independently to each other. Under the name of decision management, we group all the technologies that help organizations to automate all those simple but plentiful granular decisions and calculations that businesses often codify into decision tables, decision trees, or business rules.
Decisioning technologies to the rescue
Decisioning technologies can be used here. But one can use a database programming language or a general-purpose scripting language to automate the loading of data from the data sources into the raw data layer if the original format is kept. There is no real value using a business rules engine to do this straightforward job. The value of using a decisioning technology starts at the integration data layer, where attributes of objects are grouped together to form an updated version of an existing attribute or a new attribute.Take the example of customer data from two different databases. Suppose that customers have their home address and business address in a first source database, and that they only have their home address in a second source database. Suppose also that the format of the addresses is not the same in the two databases. What should the integration data layer hold? An address? So, which one? And what format? Two addresses? So, what professional address to put for these customers who are only present in the second database with the home address? These questions can be easily answered through decision rules.
Now, suppose at the aggregate data layer, we want to add an average turnover with a client that buys from two business units. Here again, calculation rules can be easily used. One can use SMARTS’ look-model engine to automate such calculations.
SMARTS
SMARTS is our all-in-one low-code platform for data-driven decision-making. It unifies authoring, testing, deployment, and maintenance of micro-decisions and micro-calculations described in this article. SMARTS comes in the form of one product, four capabilities:- A decision management platform that spans the entire life cycle of decisioning from modeling to deployment.
- A low-code no-code environment in which users express decisions and calculations through point-and-clicks and web forms.
- An AI & ModelOps environment that covers the full spectrum of ModelOps from importing existing models, to defining new ones, to executing learning tasks.
- A real-time decision analytics environment to manage the quality of decision and calculation performance.
SMARTS has been extensively used for decisions and calculations in finance, insurance, healthcare, retail, IoT, and utility sectors. To learn more about the product or our references, just contact us or request a free trial.
Wrap-up
- Data marketplaces promise to change the way data is consumed by businesses. Contrary to data warehouses, they are more complex to build and therefore deliver on their promises.
- Decision management technologies simplify data integration, aggregation, and enrichment when building and running data marketplaces. They make decisions and calculations explicit and therefore easy to change whenever situations change.
- SMARTS supplies multiple graphical representations* and engines to make such transformations, integrations, and enrichment easy to design, implement, test, deploy, and change according to situation changes.
* The following table, tree, and graph show three different representations of the same decision logic so that developers can use one that they are familiar with or that best fits the task at hand.
About
Sparkling Logic is a Silicon Valley company dedicated to helping businesses automate and improve the quality of their operational decisions with a powerful decision management platform, accessible to business analysts and ‘citizen developers.’ Sparkling Logic SMARTS customers include global leaders in financial services, insurance, healthcare, retail, utility, and IoT.Sparkling Logic SMARTSTM (SMARTS for short) is an all-in-one low-code platform for data-driven decision-making. It unifies authoring, testing, deployment, and maintenance of operational decisions. SMARTS combines the highly scalable Rete-NT inference engine, with predictive analytics and machine learning models, and low-code functionality to create intelligent decisioning systems.
Hassan Lâasri is a data strategy consultant, now leading marketing for Sparkling Logic. You can reach him at hlaasri@sparklinglogic.com.