Are algorithms too complex?
It is no news that algorithms and mathematical models have become a force that shapes (and sometimes drives) our choices these days. From the results from your favourite search engine, the models that are responsible for most of the transactions in the stock markets, the pricing in airlines and uber or our suggestions at Netflix or Amazon, algorithms are everywhere. If you like TED talks and you like podcasts then, first check Pint of Science. Our podcasts are more about science and less about entertainment and design (maybe more T and little ED). But if you have spare time then check the TED podcasts at NPR. One of the more recent ones is called Solve for X and includes excerpts of a talk by Kevin Slavin warning us about the increase in the complexity of the algorithms that shape (and will soon rule) our lives.
And here is the thing: as a computer scientist and mathematical modeller I believe in the potential that these algorithms have to change our lives for the better. But I am also quite concerned about increasing the complexity of certain algorithms when they are effectively black boxes. As an example of this problem, Slavin refers to a recent and massive market crash at the NYSE that was caused by one of these algorithms. Having a complex algorithm in charge of, say, Netflix recommendations is not a big deal: it is not the end of the world if some of us get recommended Wolf of Wall Street when we would rather watch Margin Call for instance. It is a different matter when an algorithm can make billions of dollars disappear from the stock market or, closer to home, determine (or guide) the clinical strategy for a patient at Moffitt. In those situations we need algorithms and models whose complexity can be understood so that when something wrong happens (as it is bound to happen) then we can learn and adapt the algorithm accordingly. Modelling is not about capturing all the complexity of a system (be it the stock market, our your taste in movies and music) but about having a framework in which to place all the key variables. This allows us to get not only predictions but also understanding which is a lot more powerful.