ML
Data vs. knowledge in automated decision management — Why not both?
In the tech industry, we also have our well-known “Coke vs. Pepsi”, “Avis vs. Hertz”, or “Mac vs. PC” debates. In the automated decision management category, the question that keeps coming up is “data vs. knowledge.” The aim of this blog post is to show that from a practical point of view, data and knowledge can be found in the same application. To do this, we will show it with SMARTS, Sparkling Logic’s decision management platform that allows users to combine data and knowledge without them entering the “data vs. knowledge” debate.
Origin of the debate
The “data vs. knowledge” debate dates to an old debate about whether knowledge about a subject should be hand coded or machine learned. A first camp of researchers and practitioners sought to encode this knowledge in the form of rules and an inference engine that runs on these rules to supply answers to user questions. A second camp sought to develop programs that learn from available data using statistical methods to generate models that can make predictions from unseen data. At Sparkling Logic, we support a pragmatic approach that consists in using data, knowledge, or both depending on the problem and the situation at hand.It is all about the situation
There is no such thing as a stand-alone decision management application. It is often built with the purpose of being integrated into a larger system for loan origination, risk management, product configuration, or other similar applications. As I wrote before, there is no one single approach. It is all about the situation.Data is everywhere, easy to collect, organize, and transform into predictive knowledge. So, if you have a lot of data, it may be better to build your decision management application around that data if the new observed data does not deviate too much from the old, learned data.
On the other hand, when you have knowledge whether in the form of rules or procedures, it is better to build your application around this valuable knowledge if it is easy to capture and code into the application.
If you have both data and knowledge, why not using the two, when you can do so in a modern decision management platform such as SMARTS, the subject of the next section.
The SMARTS way
SMARTS is a decision management platform that enables creating, testing, deploying, and improving automated decisions in an integrated platform. I will not detail it here, but you can find a brief overview of SMARTS on our blog page and a full description on our resources page. Instead, I will focus the rest of this article on how to use SMARTS when you have plenty of data or domain knowledge about the application you want to develop.You have plenty of data
For situations where you have plenty of data, SMARTS proposes two tools: RedPen and BluePen.
With RedPen, you write decisions in the form of rules using a use-case driven approach. A loaded data sample supplies the context for the rules and enables immediate execution and testing of each rule. RedPen mimics what subject-matter experts do when they flag decisions.
When you activate RedPen, you can pin an existing rule, a field of this rule, or a rule set and change it as if you were using a real pen on real paper. You can also create new rules with RedPen, SMARTS automatically turns them into executable rules.
On the other hand, BluePen lets you explore and analyze your data using your domain knowledge to find predictors. Then, using these predictors, you can generate a model in the form of legible rules and integrate them into your decision logic.
Using BluePen, you can engineer or change the models any time you need to. Without heavy investment in data analytics tools and efforts, you can evaluate BluePen models in simulations and quickly deploy them in the context of an operational decision.
You have domain knowledge
For situations where you have knowledge, SMARTS proposes two additional tools: SparkL and Pencil.
SparkL is Sparkling Logic’s language for writing rules in a natural language fashion. SparkL comes with everything you need to write rules —mathematical expressions, string manipulations, regular expressions, patterns, dates, logical manipulations, constraints, and much more. You can express any imaginable decision logic and symbolic computation, making it the choice for highly sophisticated decisioning applications where the conditions as well as the actions can take a wide variety of forms.
Pencil is our DMN compliant graphical decision design tool for uncovering, documenting, and sharing decisions with colleagues and partners. With Pencil, you just drag and drop graphical shapes to form a complete decision diagram. Then you add business logic to the graphical shapes and let SMARTS execute it.
Pencil helps you think about the ultimate decisions in a structured way, starting from the top-level decision to smaller sub-decisions. This iterative process is very friendly and amazingly easy to share with colleagues or partners working on the same project.
In addition to the above tools and language, SMARTS comes with a built-in dashboard to measure and improve business outcomes of the taken decisions.
You have both
If you are lucky enough to have both data and knowledge, you can leverage your models’ outputs by using it as the input to rules. For example, your loan management application could run a model that calculates a score and another model that calculates a risk and use that score and risk in a rule to calculate a price.
You can also do it the other way around, using the outputs of rules as inputs to your models that you would have trained with data. For example, your application might run a rule to classify a loan applicant, then run a model to calculate their risk of default and another model to calculate the price.
Whether you have data, knowledge, or both, SMARTS uses them as sources of information for the automation of your operational decisions.
Summary
- Data and knowledge do not have to be antagonistic. They can both be used as inputs to automate decisions.
- SMARTS is a modern decision management platform that enables their combination in an elegant and seamless way. For SMARTS, data and knowledge can be used as sources of information.
- When you have a lot of data, you can use RedPen to write rules without learning a special rule language or syntax, just starting with the data. You can also use BluePen to learn from data and turn it into rules.
- When you have knowledge, you can use SparkL to encode it into rules, from the simplest to the most complex rules that your application may require. You can also use Pencil when designing, documenting, and sharing your decisions are part of the requirements.
- Our mission is to enable customers to implement the most demanding decisioning requirements and to easily change and improve them over time. Whether you have data, knowledge, or both, we can help. Just contact us or request a free trial.
About
Sparkling Logic is a company at the forefront of technological innovation in decision management. We help businesses automate their operational decisions with a powerful decision management platform, designed for business analysts first.Sparkling Logic SMARTSTM (SMARTS for short) is a decision management platform that enables creating, testing, deploying, and improving automated data-based decisions in an integrated easy-to-use environment.
Hassan Lâasri is a data strategy consultant, now leading marketing at Sparkling Logic. You can reach him out at hlaasri@sparklinglogic.com.
Sparkling Logic SMARTSTM in 10 Questions and Answers
Sparkling Logic helps businesses automate and improve the quality of their operational decisions with a technology platform that is powerful and simple: SMARTS for short. In this post, we present SMARTS through 10 selected questions and answers.
Q&A
1) What is SMARTS?
SMARTS is a decision management platform for business analysts and ‘citizen developers’ to author, test, simulate, deploy, run, and change decisions autonomously, without involving developers or IT beyond first installation.
2) Is SMARTS a business rules engine?
SMARTS is more than a business rules engine. It integrates multiple decision technologies into the same platform. SMARTS provides eight execution engines: A decision flow engine to sequence tasks of a business process; a state-machine engine to orchestrate tasks; a rule set engine to sequence decisions; a sequential engine that either fires all or just the first valid decision; a Rete-NT engine for inference; a lookup engine for data search in large datasets; a PMML engine to execute predictive models; and a DMN 1.3 engine to execute decision models. Depending on the problem you have, you may choose one or the other, or even combine them in the same set-up.
3) What are the typical applications for which SMARTS is the best fit?
In the financial, insurance, and healthcare services, SMARTS often won over the competition for origination and underwriting, pricing and rating engines, account management, fraud detection, and collections and recovery. More generally, SMARTS is a good fit when there are a lot of decisions that are data-based, frequently invoked, and likely to change often.
4) What is the difference between authoring business decisions and rules with SMARTS and coding them directly in the final application?
You can code decision logic but you will need detailed specifications from business analysts. This process may take too much time when compared to SMARTS. And once the decision logic is coded, it becomes complicated for business analysts to understand and take control of. SMARTS targets business-critical decision logic that either implements business models, corporate policies or industry directives in a dynamic and continually changing economy. Think of all the financial, insurance, and healthcare regulations since the financial crisis of 2008 and the changes since the coronavirus crisis of 2020. These two crises are typical examples of complex situations where business decisions not only need to be implemented quickly and accurately, but they also need to change dynamically and continuously.
5) Does SMARTS come with a decision design process?
SMARTS not only supports but it also augments the Decision Model and Notation (DMN) standard of the OMG (Object Management Group). DMN models decision dependencies very well, but not decision sequencing, which is also a natural way experts use to describe a complete decision logic. SMARTS addresses both dependency and sequencing through the combination of Pencil, RedPen, and the decision flow.
6) What machine learning models does SMARTS support?
SMARTS supports the execution of 13 machine learning models including classification, linear and logistic regression, support vector machines (SVMs), decision trees, random forests and ensemble learning, clustering, and neural networks. SMARTS uses PMML, the standardized predictive model markup language, to import and execute whatever model your data scientists have built.
7) Does SMARTS integrate with business process management platforms?
Yes, a SMARTS decision service can be natively invoked by a business process like any other service. Also, for decision-centric processes, SMARTS provides an orchestration capability.
8) What is the difference between an RPA tool and SMARTS?
If you think of a process as a sequence of “what to do”, “how to do it”, “do it”, and “report it”, then SMARTS automates the “what to do” and “how to do it” tasks while an RPA tool automates the “do it” and “report it” tasks.
9) Is SMARTS cloud-based?
SMARTS was designed from the ground-up for the cloud. Whether you have chosen to host your application or use our SaaS solution, we provide you with the most modern tools. SMARTS comes in a container, ready to install on your premises, AWS, GCP, Azure, or Aliyun. Choose yours, change your mind, no need to recode to redeploy your application.
10) What makes you unique?
Our motto is “your decisions, our business”. We enjoy nothing more than helping customers implement their most demanding business requirements and technical specifications. Our obsession is not only to have clients satisfied but also to be proud of the system they built. So dare to give us a challenge and we will solve it for you in days, not weeks, or months. Just email us or request a free trial.
In this post, we introduced SMARTS through 10 selected questions and answers. If you have more, feel free to read our blog, sign up for our webinars, or contact us. We would be happy to get back to you very quickly.
About
Sparkling Logic is a Silicon Valley company dedicated to helping businesses automate and improve the quality of their operational decisions with a powerful decision management platform, accessible to business analysts and ‘citizen developers’. Sparkling Logic’s customers include global leaders in financial services, insurance, healthcare, retail, utility, and IoT.
Sparkling Logic SMARTSTM (SMARTS for short) is a cloud-based, low-code, decision technology platform that unifies authoring, testing, deployment and maintenance of operational decisions. SMARTS combines the highly scalable Rete-NT inference engine, with predictive analytics and machine learning models, and low-code functionality to create intelligent decisioning systems.
Hassan Lâasri is a data strategy consultant, now leading marketing for Sparkling Logic. You can reach him at hlaasri@sparklinglogic.com.
Software industry trends behind the digital transformation revolution
This article presents the three software industry trends driving the digital transformation revolution: DevOps, low-code / no-code automation, vertical integration with digital decisioning.
Introduction
The pandemic changed tech priorities for many people both at work and home making a ‘hybrid’ work a top initiative. Where and how we do the work accelerated the need to improve customer digital experiences and efficiency across work, shopping, and everyday chores.The data supports this new trend. The independent research firm Omdia compiled over 300 responses from executives at large companies indicated that working away from traditional offices will become the new norm. 58% percent of respondents said they will adopt a hybrid home/work. Even more interesting is that 68% of enterprises believe employee productivity has improved since the move to remote work.
Similarly, adoption of everyday on-line activities such as shopping, banking and entertainment further accelerated the pace of digital transformation. The need for improved applications increased the pressure on companies to relaunch efficient, friendly front-end customer apps with more intuitive UX. The back end now needs to support faster turnaround with the need to automate processes for the new on-line community of users demanding faster, cleaner, and more intelligent offerings.
To respond to this digital transformation, companies are rapidly adopting easy-to-use integrated enterprise software tools to optimize and accelerate development of these efficient digital products.
Several trends like DevOps, Low-code/automation and vertical integrations with integrated digital decisioning have emerged to help enterprises take the digital transformation journey faster and cheaper.
DevOps
DevOps is a software development concept bringing together historically disconnected functions in the lifecycle of the software development. Traditionally, business analysts would define the problem, developers would interpret the concept and build applications, and operations teams would test, report bugs and provide feedback. The disconnect between the functions, silo’d approach created inefficiencies, increased costs and slowed down application releases.The emergence of integrated tools and processes which integrate this multiple aspect of software development and promote collaboration between these different functions supported growth of the DevOps industry.
In fact, the market data shows that these trends are supported by the investment community and exit activity. According to Venture Beat, in 2Q 2021, Venture funding for global DevOps startups reached $4 billion and the exit activity deal value was dominated by the IPOs of UiPath (robotic process automation) and Confluent, (data / application integration platform).
Low code / no code automation
Application development is also coming closer to non-developers with low/no-code approach and automation.Software engineering, traditionally owned by IT and software engineers, has always been coveted by other, non-IT stakeholders in the enterprise. In 1991, Powerbuilder introduced a revolutionary concept of a development framework, aiming at democratizing development by allowing non-software professionals to get access to application development. Perhaps ahead of its time with clunky UX, WYSIWYG, Powerbuilder started the revolution of introducing emergence of ‘citizen-developers’, people who originally participated alongside IT in shaping the application and business models but could not code and create the applications themselves. It also introduced data integration with application logic and object-oriented concepts like inheritance and polymorphism and encapsulation, bringing software engineering to the masses.
Fast forward to 2020’s, virtually every enterprise tool platform and enterprise customer have adopted a low-code/no-code approach. The mission is the same as 30 years ago – to provide easy to use, graphical UI/UX, drag and drop concept to application development and allow business analysts, ‘citizen-developers’ and non-software engineers to create, test and even deploy enterprise applications.
Vertical integration with digital decisioning
The perennial challenge of allowing non-developers to create applications is the conundrum of how deep they can develop without coding and to what extent they can customize complex enterprise cloud applications without IT and coding.To accelerate digital transformation, enterprise software vendors are emerging mostly from the workflow / BPA world, such as Pega and ServiceNow. They are applying a two prong approach – core tool collection and vertical integration. The workflow vendors have developed (or acquired) a collection of point tools in a core-component framework. Those components typically include AI/ML, reporting, workflow, RPA (Robotic Process Automation), case management, rules engine, decision management, knowledge bases, BPA (business process automation) and process orchestration. Those components typically feature common UI and work across a normalized data model and unified architecture.
But that is not enough. To satisfy modern rapid digital transformation needs, in case of fintech enterprise customers (i.e. banks, insurance companies and financial services) also now require pre-built workflow, data and application models. These vertical templates are higher level and more specific, providing out-of-the box, drag/drop solutions like credit card operations, loan management and payment operations. Using the low-code approach, a business analyst can graphically drag/drop pre-defined steps into a loan origination workflow with pre-defined commonly used tasks, created using best practices defined by the ‘centers of excellence’. Companies like UIPath have created a 3rd party marketplace for additional steps and templates created by analysts and consultants. (Those steps could be ‘get customer data’, ‘OCR input form’, ‘scrub customer data’, authorize user’, ‘assess risk profile’ etc.).
Beyond the top level tasks, the functionality ultimately becomes more complex and the sophisticated customer needs powerful decision capabilities to introduce their own business rules and implement proprietary features. The ‘secret-sauce’, which separtes most common steps from proprietary concepts distinguishes top corporations from the competition, requires more sophisticated digital decisioning tools. These digital decisioning tools enable non-developers to customize and manage decision logic, implement AI/ML features, run A/B testing and visualize performance results on training and production data in real time.
To satisfy most common customer base, digital workflow vendors typically provide rudimentary business rules integrated in their low-code platforms and further integrate them with the downstream workflow platforms and vertical ecosystem vendors (i.e. FiServ, Jack Henry, SAP, Salesforce and FIS in banking for example).
The most sophisticated and demanding customers, however, need a more sophisticated set of digital decisioning tools like standalone professional DM platforms. To simplify and visualize this complex decision management, a new generation of low-code digital decision management platforms like Sparkling Logic emerged. These platforms integrate historical business rules engine, data and AI, demystifying machine-learning and providing low-code approach to development and monitoring of application logic performance, continuously as the business logic and training data change and drift.
The pandemic, hybrid work and pervasiveness of the cloud computing have irreversibly changed the software application development. Enterprise customers are seeking and deploying better, faster, more integrated software tools. DevOps integration, low-code, vertical templates, integrated AI and digital decisioning are becoming a new normal while defining the next generation of applications, created not only by software engineers, but by mere mortals across the enterprise.