As we get ready for deployment, let’s consider business testing. We covered QA testing a couple of weeks ago. The key difference is that we do not check that requirements are covered; we check that our requirements are correct. In other words, we focus on business performance.
Why business testing?
In the traditional SDLC (software development life cycle), as an over-simplification, you get requirements, they get implemented, and you test that they were implemented properly. When referring to your UI requirements for example, this is a no-brainer. You want a button for such-and-such functionality. It is where you want it. It does what you want. This is all good and easy to test.
Business testing would take this paradigm to the next level. Combined with A/B testing (what we call in our world champion / challenger experiments), it would aim to measure whether you are more successful at your business objectives with a green button, or a hyperlink.
When applied to decision management, business analytics become paramount. Certainly, champion / challenger experiments will take your business testing to the next level. However, just measuring your business performance is an invaluable first step. While only a subset of our customers use champion / challenger, almost all of them establish dashboards in which they can track how well they current policies are expected to perform once deployed.
Looking at the past few weeks or months of data, what is the load expected in manual process? Will changing this threshold affect my decline rate significantly?
Decision management is sometimes well defined, per regulations at times. But in many cases, it is an evolving art of making the best decisions for your positioning. If you are a conservative organization, but quite liberal with customer acquisition, you might end up at odds at some point.
Thanks to business testing, you can anticipate what your business outcome is going to be, with the assumption that historical transactions are a good prediction as to what is to come.
How do we do business testing?
Start with Data
In order to reach valid and relevant estimates, you will need data. The best option is to collect historical transactions. The volume will depend on your industry, and/or your type of application. Some customers use thousands of transactions; while others use billions. Some customers prefer recent, fresh transactions because of volatility in their space; while others lean towards extensive data sets collected through months or years. Regardless of your specifics, having historical transactions is the best thing for thorough business testing.
What if you do not have any historical transactions? Do not despair, there are a couple of options for you.
The first option is to look for pooled data. It is obviously not available for all projects. When the common good drives organizations to collaborate and make data available, you can take advantage of this golden opportunity. This pooled data can help you assess your business performance outside of your customer portfolio, as part of your business testing efforts. When considering customer acquisition, ranging from marketing to origination to fraud detection, this data can become invaluable.
Another option is to make up the data you do not have. When constructing test cases, you can introduce bias to reflect the distribution in your customer portfolio. For green field projects, you may not have any other option, and something is better than nothing. It is clear that your business indicators will not be as reliable, but they should give you a directional sense of how your decisions will perform.
Then, your KPIs
Business testing is about measuring your business performance. In that sense, KPIs (key performance indicators) constitute the foundation for your business testing. In addition to the QA statistics we mentioned during QA testing, you will need to establish what is important to track in your business.
Each organization, each project uses a different set of KPIs. It is likely that some indicators will track your business success: how well are you doing, how much $$ you make, how many transactions you approve, how many fraudulent transactions you stop, etc. Likewise, other statistics will measure your risk exposure: how much credit has been granted, how many customers will you inconvenience, how many overall transactions will be stopped, etc.
Taking business testing one step further
Now that we have decision analytics as to what is expected, the next logical step, in terms of business testing, is to measure these KPIs in your real-time environment. Are your actual outcomes close to your predictions? If they are, how would you continue tweaking your rules to improve them further. If they are not, do you understand why your current business is different from your past transactions? How can you take advantage of it?
The art of decision management turns into science with the right tooling in place.
Learn more about Decision Management and Sparkling Logic’s SMARTS™ Data-Powered Decision Manager
