Remember a decade ago when we were running SETI@home on our computers all night? I was guilty of lending my off-hours CPU time for this fun Berkeley experiment. My husband and I would join forces to help detect little anomalies… We did not take it seriously of course but we enjoyed being part of the program! It was also technically intriguing as the first large-scale grid deployment we participated in…
Time has passed since then.
Grid architecture turned into Cloud deployments for elasticity. The idea to join forces grew stronger though, manifesting itself in various ways. Collaboration and Social capabilities are revolutionizing the way people can work together. Some success stories involve better teamwork within the enterprise; others are about customer ideation. In this post, I will focus on those crowdsourcing initiatives that push the innovation outside of the enterprise, not for feedback but for actual work.
The Netflix Prize
We have all heard of the Netflix Prize but let me remind you of the premise. Netflix launched a competition in 2006 with an appealing $1M prize for the best predictor. Being a movie rental business, they differentiate from the established brick-and-mortar players by offering the service over the mail as a subscription rather than a per-day rental. The business model was a great way to break into the space quickly but they needed increased differentiation since all players could (and did) adapt to the new way of renting movies. Consumers want to watch movies but they do not necessarily know what is out there… Have you ever stared at the wall of movies at Blockbusters without a clue as to what you will bring home? With a powerful Recommendation Engine, Netflix can predict the list of movies that you are the most likely to love, based on your previous ratings. Improving the precision of this recommendation engine increases Netflix’s value-add and therefore competitiveness.
On September 21, 2009, the $1M Grand Prize was awarded to a team that could improve by more than 10% the accuracy of the incumbant Cinematch.
Academics have had opportunities to research algorithms forever of course but, in this case, Netflix made data freely available to the participants. This enabled a pragmatic effort to take place rather than just theoretical. The business objective was clearly stated in the rules: improve the prediction by 10% or more on the provided quiz sample.
This 3-year journey involved a lot of hard work from many teams around the world. It was impressive to see how close the race got, with another submission reaching the stated goal arriving just 24 minutes after the winning project. What was most impressive was the collaboration that took place. The leading teams realized that they could achieve more by working joining than competing (http://www.wired.com/epicenter/2009/09/how-the-netflix-prize-was-won/). At that point in time, dramatic improvements were achieved. This is a beautiful lesson learned that testifies of the value of collaboration!
The secret sauce for both BellKor’s Pragmatic Chaos and The Ensemble was collaboration between diverse ideas, and not in some touchy-feely, unquantifiable, “when people work together things are better” sort of way. The top two teams beat the challenge by combining teams and their algorithms into more complex algorithms incorporating everybody’s work. The more people joined, the more the resulting team’s score would increase.
— Eliot Van Buskirk, Wired
This prize was a stroke of genius by Netflix who realized very early on the potential offered by crowdsourcing. Not only could they achieve an incredible performance improvement to their algorithm, which they may not have been able to come up with ever, but they only spent $1M! It may seem like a nice price tag but if you consider that a team of 7 people works for 3 years on it, that is ridiculously cheap… What would have been the odds of hiring the right people, with the right motivation and ideas? It would have cost a lot more I am sure, and would have led to less tangible results.
More crowdsourced projects for recommendation / prediction engines?
With the success of the Netflix project, some new similar projects have bubbled up. The chances of FICO outsourcing their FICO score are pretty slim of course. Companies that use a prediction engine but do not live off of it are more likely to launch those initiatives.
Similar to Netflix, Overstock.com wants to provide better recommendations to their consumers (http://www.geekwire.com/2011/1-million-prize-product-recommendation-algorithm), hoping to increase their sales at the end of the day. They have just started a new competition with the now “standard” $1M prize. Following the Netflix footsteps, they also target a 10% improvement or better.
If you are on the lookout for a bigger prize you can also check out this other competition. Heritage Provider Network is offering a $3M Grand Prize for the best predictive algorithm can identify patients who will be admitted to the hospital within the next year, using historical claims data. In that case, data is obviously provided but no hard-and-fast objective is provided. The team with the best prediction will win the prize at the 2 year mark.
I find this trend very exciting for the Decision Management space. Collaboration can lead to great results with or without the carrot those companies are offering here. It may take a little while for companies to embrace collaboration outside of the boundaries of the enterprise for harvesting and fine-tuning Business Rules but I have hope that we are not talking about decades.
Learn more about Decision Management and Sparkling Logic’s SMARTS™ Data-Powered Decision Manager