Different
The Economist – The Data Deluge
Carlos started his talk with real-time data on the Chilean miners rescue. It is a happy ending we are finally witnessing after the longest rescue effort ever I believe. We looked at real-time tweets and stats on the volume of Facebook stories. Carlos being from Chile, he is quite proud and happy, relieved, but the point is mostly that tons of data gets generated. Data is everywhere surrounding us. And even the Economist had an edition on the Data Deluge.
Provocation?
After a decade at FICO defending the Business Rules perspective against an army of Analytics guys, claiming loud and clear that:
One ounce of knowledge is worth a ton of data
Carlos is now taking the provocative standpoint that while a ton of data might be worth an ounce of knowledge, we we are surrounded by tons of tons and more, and that is bound to have an impact on an ounce of knowledge. Okay, I am putting words in his mouth now. But technically he is indeed championing the Data camp. It looks like Day 3 at Rules Fest is actually the Analytics day with Carlos and Alex.
Impressive?
Carlos is illustrating the BIG in Big Data with examples faster than I can type. The one that we are all talking about in the Bay area is the Google car. In a previous life Carlos was involved with robotics research in the Car industry – you may want to get the details from him rather than me. My knowledge of cars is very limited as you may have noticed over past posts!
Anyway, back then, researchers worked on modeling the knowledge to allow a car to drive itself. It was just a humongous amount of knowledge that did not achieve its objective of autonomy with satisfaction.
Google did not invest much on the knowledge part. The algorithms are pretty simplistic actually… But they leveraged huge amounts of data to let their car drive itself on Highway 1. Just amazing.
What’s the point then?
Decisioning can leverage this data to improve its precision or become more “real-time” but the traditional ways of looking at systems just don’t work for this new environment. Why? Because the data volumes are growing too fast and the traditional models might be obsolete by the time they get deployed due to the nature of this explosive data. Simple algorithms that refresh themselves often are, in some cases, more valuable than the traditional predictive models.
Carlos presented last year in more details Hadoop Map Reduce, etc.
This year he is going into storing and processing challenges, covering different techniques and algorithms that can bring increased value into the Decisioning world. In particular he goes over parallel analytics on Map Reduce, Visualization, Clustering, Classification.
Carlos claims that you do not learn by listening… I think you do. Not fast but you do. Carlos adds that you learn by making mistakes. That is an interesting point actually that we should have raised in Karen’s presentation of learning by demonstration.
Making mistakes might require humans to be involved in the process. Nice follow up to Luke’s presentation. Case Management is the synergistic technology that will enable human expertise and Decision Management technologies to leverage each other. With gigantic data sets discovered through social networks, better decisions can be made – think of investigative cases like Fraud Detection. If you know known fraudsters, suspicion might go up to the roof.
And to close, this talk is obviously extreme on purpose. Business Rules and Predictive Analytics are still valuable and not going away any time soon. The point you should remember is that we need to think outside the box, we have an opportunity here to improve our decision-making capabilities but it will require us, as an industry, to open up to new ways, new technologies that can be combined in creative ways.
Damn, he speaks fast… A Chilean characteristic I have learned.
Sparkling questions too!
Learn more about Decision Management and Sparkling Logic’s SMARTS™ Data-Powered Decision Manager