Sunday, January 16, 2011

On Luck, and Scale

Bruce Boudreau’s problem is that he hasn’t kept any anger in reserve. In autocratic, people-managing positions- coaching, teaching, parenting, presidenting- it helps to have a sort of third gear of rage reserved for extraordinary circumstances. Blow the shock value of an explosive rant too early and, by the time shit gets really dire, you have nothing left with which to scare, or at least disturb, your underlings, and you’re left stamping your feet and turning bright blotchy red, screaming FUCKFUCKFUCKINGFUCKETY
FUCKBALLSFUCK while a group of sweaty, depressed men ignore you.

The Capitals, at this point in 24/7, have lost six games. Six. What is six games in hockey? Six games is not even a tenth of a season, a bare hundredth of a successful career. Win or lose, it is nothing. And yet here is Boudreau, screeching as though the rafters were collapsing around his ears and ravening Thrashers battering at the door, his players a gallery of tense jaws and averted glances, every game bringing another cycle of brittle optimism, angry swearing, incoherent recriminations, and sullen brooding. Wash, rinse, repeat.

The Washington Capitals, by any standard one of the elite teams in the NHL lo these past couple of years, have lost six games, and everyone is so absurdly distressed that, to the HBO cameras, they look like the most dysfunctional team in the League, and one is amazed they can successfully board a bus much less play a hockey game. It seems ridiculous, to make such utter despair from such a small thing, and a metaphor comes to mind about forests and trees and the importance of a little perspective. But then again, Bruce Boudreau is closer to an arborist than an ecologist, and Alexander Ovechkin more or less a professional koala. What may be true of forests generally is immaterial to them; this tree is their habitat and its health is their responsibility.

But there are hockey-ecologists out there. Climatologists, even. Geologists. People who think about hockey with the kind of long-range perspective usually associated with astronauts and minor deities. J Likens, in a study of power play shooting percentage, simulated 100 seasons. When Sunny Mehta was trying to determine whether shot-blocking was a replicable skill, he ran 10,000 mathematical years. And when the venerable, terrifying Gabe Desjardins wanted to demonstrate how the 09-10 Colorado Avalanche, a team whose true talent level had been assessed at something in the neighborhood of .450 (meaning on skill, sans luck, they should win 45% of their games) could clear 95 points in the standings, he went totally batshit with 100,000 computer seasons.

How one thinks about hockey is entirely dependent on how much hockey one is thinking about. Just as the style and content of customary hockey play is dictated by the size of the ice and the nature of the boards and the position of the lines and other things people never bother to consider, the style and content of customary hockey thought is in large part dictated by scale. And if I had to guess the single biggest difference between how the stats-faithful and the eyes-faithful think about hockey, it would not be some fundamental difference between quantitative and qualitative mindset, left-brained and right-brained thinkers, no great clash of civilizations between the scientifically rational and the hoodoo spiritualists, but simply a matter of how many games they consider a meaningful number.

Which means, in some disturbing backwards wraparound reacharound kind of way, that both views may be equally correct, about the same issue, even when they are explicitly contradictory.

Let me explain.

Firstly, there is such a thing as fate, and fate is probability. Probabilities, accurately derived and given enough time, will all eventually come to pass. It is utterly certain, more certain than the sun rising and or the tides turning, more certain than the occasionally irregular beat of your own heart. The seas may freeze and the mountains may blow away as carded wool and all the wonders of the earth be reduced to nothing more than a crippled friendless insect cringing into the barren rock, but if that lonesome roach flipped a coin head over tail over head into the arsenic and methane air, the chances of it coming down heads would still be 50-50, and if he flipped a million flips, having little else to do while waiting for certain death, almost exactly 50% would be heads and 50% tails. Everything in this whole frenetic world may change, but nothing can change that. It is fate.


(there is always a ‘but’, ahamdulillah…)

Probability takes time to become destiny. Lots of time, or more accurately, lots of trials. Probability often (although not always) has very little to say about one time. Or five times, or ten times, or fifty times. Most governing probabilities in the world (and especially in professional hockey) are marginal- more along the lines of 56-44 than 93-7. They cannot say with any authority what will happen in one particular instance. Our hypothetical post-apocalyptic cockroach would have to flip a thousand flips before the actual 50/50 chance was incarnated in literally 50% heads and 50% tails. If he only flipped six times, say, before keeling over of sorrow and toxins, he could easily flip six tails. Or six heads. Or three of both, or any combination. Some might be more likely than others, but for such a small number of trials, no outcome is all that unlikely or all that surprising. Over a small sample size, anything might happen.

So, if one is a statistician, and one is working with probabilities, one is inevitably interested in the largest possible numbers, the biggest available dataset, because if there are controlling tendencies in the probabilities, it is only in the great mass of events that they will come into force. And, if the real world does not offer enough actual events to work with, the solution is to mathematically simulate more, until you have the expected results of a thousand or a million trials.

Statisticians are in search of what they call true talents, which is another way of saying persistent talents, talents that can be quantified and replicated. Talents that emerge from the probabilities and grow stronger through more trials. It’s a kind of Platonic ideal of hockey skill, that the real powers are those powers that would emerge in an infinite season of hundreds upon hundred of games, played by the same teams of the same players in the same league. A few of these talents have been identified: shot volume, shot blocking, ES save percentage, and a few more yet that I cannot bring to mind.

That these talents are true is undeniable, and I have no quibble with the research. There is no denying that a true talent is the kind of thing one should bet on rather than against, and the identification of them is of tremendous benefit to general managers and gamblers, who make enough bets over long enough periods of time that thousand-game probabilities inevitably come into force. The problem comes when some things that are conventionally assumed by hockey discourse to be true talents- finishing, hitting, fighting, shootout scoring- don’t show up in the thousand-game simulations. Does it follow, then, that these things are false? Are all talents that are not ‘true’ in the infinite sense lies or myths? Things irrelevant to the winning and losing of games?

The problem with setting true talent as the only meaningful hockey value is that the inherent scale of the game stands against it. Hockey is played in insufficient sample sizes. The standard 82 games don’t constitute enough trials for the underlying probabilities work their fatalistic magic, but the game too brutal to be played in thousand-game increments. The human body forbids it. No league, ever, anywhere, will actually play hockey to the extent that the probabilities demand; even the 164 games that might be enough for the data would be far too much for a person to bear. The annual reset button that erases the count also erases the probabilities, forces them to restart from zero, in the no-man’s-land of one and five and ten games.

For the statisticians, everything that exists below those probabilities, everything that is not a true talent, a thousand-game talent, is luck. For them, luck is the noise that fills in all the parts of the game that are not replicable on large scales (see On Luck 1). This noise has a few names. They can call it variance or they can call it chance, and then sometimes they go metaphorical and call it a crapshoot or a coin-flip, but mostly they call it luck.

Statistical luck, however, is actually the aggregate of two different things: crude luck (unpredictable and inexplicable events beyond any human influence) and the myriad actions by which hockeyists try to manage that luck. Down at ice level, they are endlessly choosing things. Coaches are choosing line combinations and drills and game plans, players are choosing training regimens and equipment brands and habits, choosing to pinch or cover, shoot or pass, finish the hit or chase the puck, block the shot or let the goalie see it clear. They never cease choosing, and their choices are not consistent because they are responding to crude luck- the injuries, the bad bounces, the flukes, the acts of God. Luck has them constantly spinning through streaks and slumps, additions and deletions, every night another city another opponent another set of boards. Hockey is played in a maelstrom of luck, and a hockey team, like a windjammer crew battling around the Horn, is constantly adapting, shifting, and correcting, trying to make headway whether the winds be fair or foul. It can be, at times, extraordinarily difficult work, punishing both physically and psychologically, and there are situations- not all that rare- where a grim, compulsive attention to the details of the job itself is the only thing that makes the effort possible. It is the little decisions that keep the show on the ice.

It is concerning the power of these million little decisions where the thousand-game-view diverges inexorably from the one-game-view. Most of the things players and coaches do in the course of a season barely register on the statistical level. They don’t hold long enough. A line combination blossoms for a few games and vanishes, a tactic deployed against the Penguins is not even considered against the Leafs, a player is shifted up or down the lines and then to the right, trying to adapt himself each time. Beneath these things, are countless more specific details fans are not in a position to notice, the emotional and psychic terrain of the game- the little promises, losses, gains, breakdowns, hatreds and passions by which any professional conducts his profession.

The statistician looks at all of this mess and, if he cannot parse the data in such a way as to generate a persistent trend over his longue duree, dismisses it as random. But this perspective is a luxury of distance. It is not that those living, or thinking, in one-game-increments don’t want to see the big picture. It’s that they cannot. It’s structurally impossible. To do their work, to conduct their lives, to perform at the high level they do, they have to believe in the efficacy of their choices. To believe that they are not coins or dice, that it is possible for them to make good and bad decisions that can make the difference between a win and a loss. They have to believe not in what is quantifiable but in what is visible, or more accurately, doable. What might be true of thousand-game hockey is as relevant, in the middle of a shift, as what might be true of Narnian hockey, for what concerns them is not what would win some slim majority of a thousand games, but what will win this game.

It is literally impossible for the people in the fire to live their lives believing that ‘luck’ is as encompassing and powerful as the data suggests. They cannot believe that all these things around them- the equipment choices, training regimens, psychological disciplines, variant strategies- are all drowned in luck. It suggests that their decisions are meaningless. And yet, the decisions have to be made, just as life must be lived. If a player believed in nothing but an underlying talent that would inexorably assert itself no matter what, he would lapse in his mad passion to improve, and he would lose his edge and fail. He has to believe- not that every choice is always significant, or that no randomness exists at all- but that any choice might turn out to be critically important, even in a small number of instances. The fervency that makes elite players is an indiscriminate and in many ways irrational thing, but it is precisely its lack of skepticism, it’s willingness to pursue such a wide, wild variety of tactics, techniques, and tenets that defines it.

Clutch may not be a replicable talent, but the coach has to put someone out for those desperate moments, and he will inevitably choose a player who projects confidence, who has been successful in such moments before. Fighting en masse, that great aggregate of all major penalties over a half-dozen seasons, might not win games, but there is no denying that every now and then the kicking of a particular ass provides a cathartic release of tension for a struggling team. The aggregate of all shootouts may look, on the scatterplot, like a crapshoot, but when a skater first cradles the puck of the center dot and prepares his move, it might make a full point of difference whether he chooses to try five-hole or over the blocker. In all these cases, a human being must make a choice, this or that, and thinking that it’s all random anyway is simply not useful. For people experiencing hockey below the luck, there will always be a market for the myths and superstitions and gut feelings, the glancing fragmentary perceptions, the customs and rituals and lore, all those things that can’t be demonstrated mathematically. These unquantifiable ideas address situations and moments that the data doesn’t deign to consider. They provide explanations and suggestions that are applicable to single, individual moments, when the data is too frail, too marginal, and simply too far away.

The problem is that hockey will always need people thinking and operating on both scales, the large and the small. Up in the high offices, where the general manager allocates his cap space and metes out the years of contracts, he has to think in multiple seasons and hundreds of games, and he would be remiss in his duties if he allowed those large-scale decisions to be swayed by what a player did in the last game, or the last two games, or the last ten. A thousand-game decision must be made on thousand-game evidence to be made well. But similarly, when the coach is allocating his roster positions and meting out the minutes for a game, he will consider who played well in the last game or two or ten, and the way their play style matches against the opposition lines, and whether he thinks there will be a fight, and so on and so on, and most of these decisions will have little basis but the work of his own eyes and opinions, and that too is right. In a hockey team, these two perspectives are complimentary, operating on different levels of the organization, serving different purposes. The conflict comes, and comes hot, when fans and journalists try to compare lessons from the two scales as if they were coeval, with the mission of eradicating one or the other way of thinking about the game. It can’t be done. Players are always going to believe that shot quality matters, because they have to decide what shots to take from where and they know that some of those choices are better than others, and they know that sometimes whether or not they get to the right place and pull the trigger at the right moment will be the difference between two points and none. General managers are always (hopefully) going to believe that shooting percentage will regress, because no matter how hard you try to get the good shots, sometimes you’re going to meet a hot goalie and sometimes they’re going to get blocked and sometimes your defenseman won’t make a good pass and in the end a lot of shit is going to get in the way of that struggle to make the best chances such that it can’t be sustained every game season after season in any bankable way. Both are right, and both are thinking the way they must to be successful.


Anonymous said...

Having read a bunch of hockey blogs over the past few years I've been amazed at the vitriol and spite spewed forth in the comment pages about this. I'm an experimental physicist in quantum mechanics (a field which only makes any sense with averages of many thousands of measurements, yet requires you to does these measurements on individual things) and we constantly have to cope with the difference between the large scale ensemble average -quite often it is physically impossible for this average to occur in any particular event-and the completely undefined individual event itself. Turns out it's quite important and if you can explain it you'll get a whole bunch of Nobel prizes!
I've been longing to write this post for ages but sadly don't have the writing skill........ and I'm lazy
Thank you

Doogie2K said...

I don't think I could express how I feel about it any better than this. And honestly, I think a big part of the problem is that word, "luck." Because "luck" encompasses both luck and...let's call them "intangibles" (all the crap that matters but can't be quantified by us as fans), people unfamiliar start howling about how it can't be luck, because this guy's clutch, and that guy has changed his mindset, and these guys have good chemistry*. The truth is, these things all matter in the short term -- given how scoring chances and Fenwick/Corsi tend to fluctuate with each other, obviously shot quality matters in the short term, and affects S% fluctuation quite a bit -- but in the long term, it all kind of washes out, because hockey and the human mind are cyclical, and given enough time, everything will balance out. If only there were a little more understanding in the orthodoxy that just because something looks random, doesn't mean it's not controllable or explicable over the short term. (I read a great article in SciAm Mind a while back about how your perception of the size of a goal will change with success or failure to hit it; the study was done with football, but I'm sure it'd work with hockey, too. Another thing that would explain part of the fluctuation of S% over time, and streaking/slumping behaviour that will probably be shat upon by the intelligentsia as unprovable in the math.)

* - As I think of it, "chemistry" is nothing more than the ability to understand/predict a linemate's tendencies and use them to find him in an advantageous position, and it's a function of hockey sense and familiarity. It's why we used to call Fernando Pisani "Instant Chemistry": he just made smart plays and made it easy for his linemates to find him to either make the safe out or to get a good offensive chance. Or why Gretzky in his prime could turn any plug you stuck him with into a goal-scorer.

Hawerchuk said...

Great piece! On thing though:

1) Fighting has quantifiable value:

2) There is real shootout talent:

3) There is such a thing as finishing ability:

4) Hitting, for its own sake, does little, but separating the player from the puck has huge value.

5) I believe streakiness does exist in shooting and goaltending, though I haven't quantified it.

But I take your larger point...

E said...

anon- validation from science? huzzah! of course, for all i know, you're actually a delusional sheep-shearer with a shelf full of brian greene books, but fuck, it's not every day i get physics on my side, so i'm going to go with it. thanks!

doogie- thing is (and i'm still wrestling with this, and fear i don't have the analytic chops to work it out), there's still this cloudy area in the middle, yes? somewhere between one-game-truth and thousand-game-truth there's a point where this great 'washing out' of incidental factors takes place, and figuring out where that happens and why and to what extent it's manipulable... that's where the money is. prolonging the life of fundamentally unsustainable things. blah. like i said, still working on it.

gabe- that is totally five things, not one thing! but thank you. actually, this is a big problem i've been running into lately, which is that it's bloody hard to retrospectively figure out exactly where your collective wisdom stands on these various issues. even if one has been reading quite a bit of the work as it comes up, finding it again when one wants to reference it is a search-engine nightmare. usually i just ask julian, who has a great memory for when-he-read-something-on-which-blog, but it turns out he can only do that for things that involve the oilers. y'all need an annotated bibliography! (not saying you personally should do it, just that it should be done one of these days.)

Hawerchuk said...

Well, I think there's some disagreement on some of these issues. Shootouts, for example - no individual shooter has had enough shots and enough success to establish his skills to 2 sds. I took a shortcut which can be disputed.

Anyways, I think you might like this:

It only points to my work. There's some genius stuff out there by at least a dozen other guys...It would be an immense project to assemble it...Maybe we need to ask everyone to put together their own links.

E said...

thanks, that is in fact tremendously helpful. i'm actually considering doing a few 'annotated bibliography'- style posts on some of these themes, trying to get a handle on where the work stands now and how it arrived there. i don't expect it would persuade all the haters, but its the sort of thing that's useful for people of my ilk (i.e. the fairly intelligent but tragically humanities/social science-educated).

MathMan said...

Something you made me realize.

The training regimens, the line combinations, all the gestures and the care that players and coaches feel they must do and do well to be successful... these things DO matter on the statistical level. They do matter because all the teams do it, and one did not, then the vast parity of the league that causes so much to be drowned in luck would disappear and you'd immediately start seeing that team consistently doing poorly.

Hockey is drowned in luck not because some of those things we feel to be real skills aren't real skills, because they're just really random chance. Hockey is drowned in luck because the teams' aptitudes in those real skills and talents and all that hard work are balanced on such a razor's edge that they largely balance each other. Norway and Canada may be a mismatch at the WJC, but that kind of ability gap simply does not exist in the NHL where the best team in the league will win less than 2/3rds of their games. And if you're not at the top of your game, even a little, then it DOES hurt your odds, but the teams make such efforts to be at the top of their games that this, too, ends up getting drowned.

There's so much parity in hockey that winning often becomes more luck than, not ability, but difference of ability. "Anything could happen", "anyone could win any given game", that "all you need is a few breaks", that "getting the bounces" can make the difference between a win and a loss, that a "hot streak" can carry you far in the playoffs. Even the day-to-day people recognize these things and repeat them often.

They (and their fans) understand the importance of randomness in their lives, but "luck" has become something of a dirty word, as if it is implied that the winner has no merit for victory. I think that's why talk of "luck" makes people defensive -- there's the connotation that there's no skill involved, whereas the reality is that without that ability that makes the NHL such a high-parity league, no amount of luck could possibly save you.

E said...

hah, to a certain extent you've anticipated parts 3 and 4 of this venture, so i'm not going to say everything i have to say at this point (otherwise it'd be another five pages). doogie has an article about this very thing, though, a while back on his site.

the short version is that i'm not convinced we can be sure we're at the 'end of history' in terms of hockey training/strategy. we know that there's an evolution to the game in these areas, and it is true that hypothetically there will be an end point where everything that can be done to improve has been done and nothing is left but luck. but i'm not sure we can know when that point has been reached, and while i know that most players/coaches acknowledge the role of luck in the game, they clearly haven't reached the point where they believe that it's the only thing that has a role.