Intelligence, Prediction and Crowdsourcing

How good are you at predicting the outcomes of political events?  Would you have predicted that the Arab Springs were going to occur?  That September 11 was going to happen?  That the US would (eventually) capture and kill Osama bin Laden?

How about if there were a utility which collected and combined the predictions of several individuals?  If we were to aggregate all of your predictions, and all of your friend’s predictions and those of experts all over the world, would that be a better way to predict what might be likely to happen?  Is a group of experts better at predicting events than a single expert?  A statistician would suggest that by having a group of predictors, it might be possible to realize a MEAN (average) or MEDIAN prediction (the one found directly in the center of all predictions).  For example, if we were to ask a question like:

How much longer do you expect that China will remain a one-party state?

 you might receive a number of predictions which ranged from an optimistic ‘six months’ to a pessimistic ‘fifteen years’.    If you asked enough people, you could expect that the predictions would thus form a bell curve, with most people making a prediction which is somewhat conservative and falls squarely in the middle, while only a few individuals would make the sort of extreme predictions which would fill in the ends of the curve.  This article explains what the baseline requirements are for group or aggregate prediction to work.  (It also explains why ‘polling the audience’  and phoning a friend works so well on “Who Wants to Be a Millionaire.”)

New developments in information technology have allowed us to begin experimenting with this relatively new idea of crowdsourcing.  Social networking technologies allow individuals to work together on group projects – like collecting information, compiling information or making predictions – on their own time from the privacy of their homes.  Wikipedia is an experiment in crowdsourcing, as articles are written together by groups of analysts who serve as checks on one another’s work.  Foreign Policy magazine has offered some wonderful examples of the ways that individuals all over the world worked together to pass on, compile and share information on the location of survivors after the devastating earthquake in Haiti.

Recently, I was invited to join an experiment in predicting world events which is being put together by this organization. Feel free to visit the link, to look around and even to apply to participate in this project.  So far, I’ve only participating in making a few predictions but I’m starting to see some problems with the model.  My concern is that the most enthusiastic participants are likely to be people who feel very strongly about particular issues.  Therefore,  I’m concerned that the people who participate the most may be the least representative.  People who are moderates might be less inclined to give their opinions because they feel less strongly about the issues.  It will be interesting to see how this project works out in the long term.