I’ve been meaning for some time to write about swarm intelligence.
The basic insights have been around for at least 30 years, and, since then, the ideas have percolated into awareness and are now appreciated outside the scientific community.
One excellent summary can be found in Len Fisher’s 2009 book, The Perfect Swarm. I read this a while ago, and took notes, meaning to share my thoughts here, so stay tuned!
I am interested in many of the ideas associated with the study of collective behavior, because they tie in with other concepts that I find fascinating. A key theme in this work is self-organization, which has been used to explain subjects as diverse as crystal growth, sand piles, evolution, and human social organizations.
And, oddly enough, gambling. Or, perhaps, more kindly put, investing. I’ve been interested in both aspects of risk-taking for about as long as I can remember, and they are identical in one important aspect: trying to understand the risk associated with an uncertain outcome, and deciding if the price on offer is a fair one.
A couple of years before Fisher’s book was published, the National Geographic Magazine published an article entitled Swarm Theory that outlined the basic ideas in the field. They did get one thing terribly wrong, however.
It appears they may have lifted this example from The Wisdom of Crowds, by James Surowiecki (which I have not read, presuming that I understood all the points he made in the book, from what I’ve read about it). If so, then they repeated his error, but wrong is wrong…
My friend Bill Ziemba studied (back in the 1980s) racetrack betting, with an eye to discovering whether parimutuel odds were unbiased in their prediction of a horse’s chances of winning a race. He found, perhaps surprisingly, that they were. I read his book, and he sent me several academic articles (both published and unpublished) on the topic, as well as some large data sets and computer code. One thing he had discovered was that, although win odds were efficient (i.e. unbiased), show odds were not; and therein lay a profit opportunity.
We’ve not been in touch for some time, but I do notice on his website (the link just given) that he is in the process of updating his work, this time for exotic bets. I’ll be interested in seeing his results.
It may be that Surowiecki used Ziemba’s work in describing the behavior of horse race bettors, but in any case, the National Geographic description was dead wrong.
Why are [bettors at a horse race] so accurate in predicting the outcome of a race? At the moment the horses leave the starting gate, the odds posted on the pari-mutuel board, which are calculated from all bets put down, almost always predict the race’s outcome: Horses with the lowest odds normally finish first, those with the second lowest odds finish second, and so on.
Wrong.Wrong.Wrong.
This is not what Ziemba found, and the statement contradicts simple statistical evidence, as well as anecdotal observation by anyone who has ever paid attention at a race track.
The favorite, or “chalk” in each race (the horse with the lowest odds) will win less than half the time. They do not “normally finish first”! The odds against the outcome described in the quote here are long indeed; in all the years I’ve been betting on horses, I don’t remember it happening even once, except perhaps in a race with very few runners.
What Ziemba found, as stated above, is that parimutuel odds are “efficient” — which simply means that you cannot use the odds as a single piece of information to make money. A fair coin has 1 to 1 odds of coming up heads when it is flipped. These are efficient (fair) odds, and you cannot make money by betting for heads or tails, over time. You might win several times in a row, but over enough trials, you will break even.
Similarly, a horse that goes off at 10-1 might win, and you would get back your original bet and an additional sum of 10 times what you bet. But if you bet on 10-1 horses over a long string of races, you should expect them to win only once in every 11 bets, and you would break even. (Except, of course, that the track deducts a fee from the parimutuel pool before computing the odds, so you will be out that fee of 15% or whatever is being deducted.)
Such are the complexities of emergent behavior arising from complex, self-organizing systems. Care must be taken not to view the system as having a central intelligence. Adam Smith had it right when he said there was an “invisible hand” at work, although his conclusions were not always correct interpretations. When people talk about “the market” they are describing the properties of an emergent behavior arising from the actions of individuals. A subject for another essay.
