From Standard Deviations, by Gary Smith [Chapter 8: When You're Hot, You're Not]:
This is one of my favorite chapters so far (although that's not saying too much, because each chapter has been a gem!) in that he delves further into the story-telling aspects of human beings explaining random patterns as if there was some method to the madness.
Here are some highlights:
1. We see some pattern and we discount that it can't be random. Instead we ascribe a story to it: like hot hand theory or stock-market fluctuations.
For e.g., take the sequence
x x x x x <= five x'es in a row!
That CAN'T be random! In fact, Smith does coin toss experiments and shows that such patterns can quite easily occur! Alas, in a sequence of 20 coin flips, the theoretical probability of getting at least 4 heads in a row is .768! so there's your hot streak!
Of course, there could be some process that's generating the five or more x'es purposefully. However, that's not the point. The point is, that we don't know what's producing it. Even a long string of such x'es can be produced at random and human beings don't account for that. They immediately want to tell a story about how those five x'es were created. We shouldn't be so quick to dismiss that the explanation is plain old randomness.
2. If we were asked to construct something that looked/seemed random, we would never create:
x x x x x
or
o o o o o
or
x o x o x
because these have patterns and there's NO way that randomness would produce that right, RIGHT?
So similarly to point 1, where we are predisposed not to recognize/treat a pattern (5 x'es in a row) as potentially being generated at random, that predisposition also tends to block/obstruct us in creating strings of random patterns because we would never even think of including those with patterns (5 x'es in a row) because such sequences are too ordered to be random (right? RIGHT??)
Smith goes on to talk in more detail about hot-hands. He says that hot-hands (and cold hands) may actually be a real phenomenon but may not have a large enough effect to measure precisely in really popular sports like basketball because of many confounding factors. (e.g., player takes shots from different angles/positions, confidence of player is higher/lower, person guarding them may be more/less aggressive, etc.); The overall message is that:
a. The hot-hands phenomenon is hard to measure (especially in popular sports with lots of confounders); However, he was able to measure it with horseshoes and bowling and found results that had significance. So the phenomenon is real, however it's not huge.
b. What the public normally sees in popular sports (basketball, football, baseball, etc.) where a player gets 'hot' can easily be the result of randomness (see the above e.g. with the five x'es in a row). Sure, it's a nice story to say there is a hot-hand at work but it's only a story and makes for good TV!
