The Hanover Collaboration

Cosmo apparently doesn’t know much about credit/debit cards. Namely that everything he purchases from a merchant that accepts credit/debit cards costs as much as 3% more than it should.

(Yes, you read that right. Merchants often raise all their prices by 1-3% to cover the fees they pay to provide the optional convenience of credit/debit purchases. If you buy everything in cash, you’re still subsidizing other people’s use of credit/debit cards.)

Your government is trying to save you money, Cosmo.

Thank them.

I didn’t get to watch the game, but the US men’s soccer team beat Honduras over the weekend by a score of 3-2, which assures their participation in the 2010 World Cup in South Africa.

Color me unimpressed. Honduras, media outlets are quick to point out, has been tough to beat at home. Of course, they really only play CONCACAF opponents, particularly at home.

And judging by the highlights, the US only barely edged out a win. Honduras should have tied the game in the 87th minute, but instead sent a PK over the cross bar. Not to mention the fact that Casey’s first goal looked, well, pathetic. For Honduras, that is.

It is stunning to me how much denial exists regarding the quality of soccer among CONCACAF nations relative to the rest of the world, Europe in particular. One really needs to keep in mind that Honduras (indeed, even Mexico these days) is just not a very strong force in international soccer.

This is perhaps an unfair comparison, but the Honduras men’s Olympic team lost all three of its games in 2008 (3-0 to Italy, 1-0 to Cameroon and 1-0 to South Korea). And this is the country we are excited about barely beating on the road?

Perhaps the most distressing fact about the team? Their midfield is so shitty, a player who has logged zero minutes with the US team, indeed hasn’t (to my knowledge) even practiced with the US squad, or any squad because he’s still rehabing from surgery is generally considered a likely starter as soon as he’s available, and has effectively been penciled into the lineup for the World Cup.

And this fact is referred to as a positive sign for the US squad going forward. Good grief.

Don’t get me wrong; if Jones is what people say he is (I’ve never seen him play) then he will surely help. But doesn’t this suggest something about the level of player we have right now? That some guy we’ve never seen play on our squad and is currently injured is considered a potentially massive upgrade in player quality?!

If I were Ricardo Clark, or Michael Bradley, or Benny Feilhaber or Sacha Kljestan, I’d find this situation fairly embarrassing.

This from Cosmo has me a little confused, because I generally consider him to be way more knowledgeable on the intricacies of OS X than myself.

And picking on Cosmo when I think he’s whining excessively is a sort of pastime of mine. So I would suggest to him scrolling down in the left pane of the open file dialog box. If he did, he ought to see a Media menu that expands to include a subset of Music, Photos and Movies.

It’s in every app that I’ve tried on my computer. I didn’t look carefully, but I’m guessing Cosmo was trying to upload an image from Chrome? If Google hasn’t implemented this feature, that’s their fault, not Apple’s.

The United States men’s soccer team upset Spain yesterday and unsurprisingly it has caused quite a stir. They played pretty well and certainly deserved the win. However, it is my nature to be both contrary and pessimistic, I thought I’d offer some thoughts on the game.

I haven’t actually watched Spain play all that much so my observations on their performance are largely speculative. But to me, they seemed rather disorganized in back, at least early on. I’m not sure if that was a result of their back four not communicating well, or if it was some defensive midfielders not doing their job.

The US played pretty well for about 35 minutes. There were about 4-5 players who ran around like freakin maniacs in midfield pressuring the ball, scooping up errant passes etc. They were (to my eyes): Bradley, Dempsey, Clark, Davies and Spector. Davies and Spector are actually a forward and defender, but they were dropping back and pushing forward (respectively) quite a bit. During these early minutes, the US did what it does best: run around like crazy and be physical.

Spain started pretty flat and continued to be out hustled by the US for much of the first half. Altidore’s goal was legit, though he had a significant piece of help from the Spanish defender marking him who decided to try to climb over Altidore’s back rather than simply trying to keep him from turning.

For me, the second half was essentially a complete disaster for the US, despite Dempsey’s goal. I’m really curious why the US retreated into their defensive third for the rest of the game. Did Coach Bradley tell them to do that? Or was 35 minutes all they could muster physically at that intensity level?

Either way, hunkering down behind a 1-0 lead for 55 minutes against a team like Spain is, generally, not a recipe for success. Every now and then it’ll work, but it takes luck and your opponent has to have a bit of an off day.

I would have preferred to see the US at least try to maintain the intensity level they started with in the second half. It might have been riskier, and might even have cost them the game. But it would have been a good opportunity to see just how long they could keep it up.

As for individual performances, I was most impressed with Bradley and Spector. Mostly that’s based solely on the effort and intensity they put in during the first half. Bradley should be sent to bed without supper by his Dad for the red card. Unthinkably stupid foul. Which, sadly, has become a trend with this US team.

Altidore continues to fail to impress me. He does only a few things, and not particularly well. He sits up front and waits for the ball to be served through the defense. He gives chase and either is unmarked (rare) and shoots (meh) or he automatically falls to draw a foul. I was much happier with Davies, who at least displayed a wider toolkit, coming back to help develop plays in midfield.

Howard did well, as usual. Dempsey I can live with, if he is used properly. He should really just be a defensive midfielder and not push forward much.

Donovan? Meh. He’s been billed as the Savior of American Soccer for so long now, I always am disappointed when I watch him. He’s no Claudio Reyna, that’s for sure.

It was nice not to see Beasely out there and Jay DeMerit needs to go. Maybe Heyduk will be back soon.

All in all, it’s nice to see the US get a good result like this. But it doesn’t erase from my memory the travesty that was the Costa Rica game, or their uninspired win against Honduras, or their lopsided losses to Italy and Brazil.

As one who is nearing the end of their graduate studies (the defense is being scheduled, the dissertation is basically written, etc.) I find myself spending a lot of time ruminating on just what a horrible experience it has been. (Don’t pity me; I am solely to blame.)

One aspect of this regular cogitation is focused on the fact that PhD suffering, (which comes in many forms and can be experienced differently by different people, so YMMV) is very difficult to relate to others.

Some well known attempts can be found at PhDComics. Indeed, some of their most popular comics are attempts at capturing the desperate ennui of grad school: see here, here, here, here and here. These are pretty good, although I usually regard PhDComics as a fairly bland and predictable form of humor. They usually evoke not much more than a grunt of familiarity from me.

However, the brilliant musings of Scott Eric Kaufman at Acephalous are pure genius. If you are not already reading his blog, go there now. He’s a humanities guy (literary theory and rhetoric in visual media I think) so I frequently don’t follow the academic stuff. But the guy is the only person I know who could give Cosmo a run for his money as the Toughest Writer Alive. The guy can write.

And he has written depictions of life as a PhD student that, while topically distinct from my own experiences (statistics), honestly feel like he has been secretly videotaping my life. And what a sad, depressing life it has been for the last few years. Don’t believe me? Read these four posts in order. Now read this.

Done? Good. Now do you see? Do you? The horror! The horror!

I need some help. I’m teaching a class called Introduction to Probability and Statistics. It is intended as a first course in prob/stats for math and CS majors, although there are always a smattering of students from other departments as well.

In addition to the usual stuff, the students are supposed to gain some basic proficiency at “basic tasks” in R. Usually, this means learning enough to answer “What’s the probability of…” questions using simulation (i.e. a for loop and some random number generation). I also use this as a tie-in to non-parametric estimation methods (i.e. bootstrapping in the stats half of the course).

The point is less the actual tasks they’re doing in R as exposing them to a major tool in statistics these days. So I’m essentially teaching a computer language.

Sadly, about half (out of 30 or so) the students have never been exposed to computer programming at all (!!!) and the other half have 2-5 languages under their belt (usually a subset of Java, C(++), Perl, Lisp and Python).

There are classroom management problems here, of course, with such a bifurcated class. But my real problem is just teaching the beginners how to get around in R. I just don’t have any experience teaching people stuff like this. It feels overwhelming for me (I don’t know where to start) and I’m sure feels overwhelming for the beginning students as well.

I’ve tried to boil it down to a simple list of concepts and commands like vectors, assignment, boolean comparison, for loops, and assorted useful commands (sample(), which(), any(), sum(), mean(), sort() and 2-3 others).

I feel like I’ve explained the following concepts, but the students’ behaviors are suggesting otherwise:

• The idea that expressions/functions return a result that need to be assigned somewhere for later use.
• The idea that variables (I try to avoid the word “object”, though maybe that’s a mistake) need to be created before they are referred to or used.
• The (syntax) idea that commands (i.e. functions) operate like mean(sort(x)) rather than like x sort() mean().

I’m not kidding about that last one. I’ve corrected that syntax error several times now, and it still crops up. Indeed, the very notion that R could possibly be serious about expecting code to be written in a certain way seems outlandish to a lot of students.

In one case, I tried to explain the syntax of for loops, which boiled down to saying that it just repeats the lines of code inside the {}, n times, incrementing the variable i by one each time. Several minutes later I helped a student who was simply incredulous that R was actually pissed that he hadn’t typed that final }. He had no problem with the for (i in 1:n) part, but was just flabbergasted that R could be so picky about a silly little curly brace.

Sigh.

Anyway, I’m feeling like a bit of a failure here. I’m sure it’s not entirely my fault, but still, I have zero experience teaching computer languages. Does anyone have any advice or experience that I might find useful?

Consider a common computing task: writing a document. We’ll leave it that vague.

Now imagine you’re writing software to allow a user to write a document. Is it possible that there are really only two strategies that are feasible? Namely, to make the user’s life as difficult as possible in one of two ways: (i) create software that constantly attempts to anticipate the actions and desires of the user or (ii) create software that must be explicitly instructed by the user to accomplish even the simplest and most common of tasks.

Both strategies make the user’s life difficult, in some sense, right?

What I have in mind here is Microsoft Word (i) and LaTeX (ii). Neither are meant as criticisms (although I generally despise Word). I haven’t really used genuine word processors in years, so I sometimes wonder if I’d hate OpenOffice or Pages just as much.

Anyway, that was just a random thought.

I recently finished reading The Black Swan by Nassim Taleb (thanks Dad¹!). I’d certainly heard a lot about Taleb, mostly in regards to our recent national financial problems. I have even seen some short interviews he’s done, and he has generally struck me as a smart, albeit overly animated, guy.

I was very disappointed to discover that his book (I haven’t read anything else of his) is really quite bad. I’d call The Black Swan an interesting topic in search of a well researched book. Andrew Gelman, a statistician I have a lot of respect for, was fairly charitable compared with the reviews he links to. That said, after reading the book, I’m much more in agreement with Lindley and Robert.

First I should say what I liked about the book: the observation that “standard” statistical models are increasingly inadequate in the face of modern data is important and timely. This doesn’t mean that the “old” Gaussian methods are useless, just that many of the modern, sexy problems that people want to work on don’t fit that paradigm very well. This is related to my earlier musings. Statistics, as an academic field, is partly to blame for the broadside attack it gets from people like Taleb. The field has, at times, become too enamored with what works mathematically, rather than with models that are useful in the world.

This has led to a growing divide between (1) Statisticians and (2) People Who Do Statistics. The problem is that (1) is contained in (2) but (2) is growing much faster than (1). So the field of statistics has acquired a severe public relations problem. Mainly this comes in the form of not understanding and communicating the level of uncertainty in a model (including caveats related to model assumptions and model shift). Taleb is correct that this popular misconception is a major problem.

However, I think that Taleb gets the cause of this problem exactly backward. He thinks the problem is too many narrow minded, formally trained Statisticians who think the whole world obeys their neat little Gaussian distributions. My experience with Statisticians is exactly the opposite. They are, in my experience, exceedingly cautious. Every time I have observed (or participated) in a statistical consulting exercise, the client spends a lot of energy trying to force the statistician to give them The Answer. The statistician spends a lot of energy trying to convince the client The Answer doesn’t really exist, and instead we have to settle with A Likely Possibility (?) and lots of caveats in the footnotes. (Frequently, I’ve had to explain to people that we can’t really conclude anything from the data. When I say that to Statisticians they nod and shrug. When I say that to the owners of the data, they often freak the fuck out. So how is this the fault of the Statistician, again?)

Most of the “reckless” statistics that I’ve seen has been perpetrated by people who are doing statistics, but aren’t really Statisticians, in that they do not have a graduate degree in statistics. Instead, they usually come from another field that involves a lot of math. Sometimes they have taken graduate level mathematics courses, sometimes not. I’m not claiming that this is a statistical knowledge issue. Instead, I think there is a cultural phenomenon here, where Statisticians, as a group, have developed a culture of cautious data analysis, but have not become very good at spreading this trait except through PhD advising.

I also like descriptions of the “impact of the highly improbably”, to steal from his subtitle. It is an interesting way to think about the world around us. So many of those stories are interesting and thought provoking. And there are certainly lots of stories of improbably shit happening in this book.

Anyway, on to my actual complaints. I could list a lot, but I’m going to focus on only a few.

Many of the modeling/prediction challenges that Taleb raises are indeed interesting. It would be really cool if someone wrote a book about all the research (past and present) that’s been done to attack these issues. Instead, we get Taleb waving his arms wildly in the air as though he’s the first person to realize that, say, the Gaussian distribution isn’t always a good idea, or that sometimes the underlying process you’re modeling can change, rendering your model useless. These are not, as Taleb would have us believe, new and revolutionary ideas. They are old ones that have received lots of work and attention by very smart people. Taleb either doesn’t know about this history, or doesn’t care. It’s true that we arguably aren’t very far along in solving these problems and I’d love to read a book where someone went and interviewed a lot of people who are actually tackling these issues. Taleb apparently couldn’t be bothered.

Taleb attacks the entire notion of inductive reasoning. Sadly, he completely misses the point. Look, suppose you’re busy predicting the future based on your past experience and things are going swimmingly, i.e. you’re getting things basically right. Now suppose either (i) an aspect of the world is suddenly revealed to you or (ii) an aspect of the world actually changes. Both instances will likely cause your model to be (very very) badly wrong.

But in what sane world is this a criticism of inductive reasoning? The fact that you can build a model that does really well for a while, and then the world changes and your model sucks doesn’t reveal any magical flaws in inductive reasoning or modeling in general. That’s just life. Again, Taleb is right to point out that we should do a better job of educating people of these issues, so at least he’s got that going for him.

I won’t make this post any longer with detailed descriptions of how Taleb displays his utter ignorance of statistical theory and practice. It suffices to say that, if you feel it necessary to read The Black Swan, you should not bother reading Chapters 15-19. If you know a lot of statistics you will feel unclean. If you don’t, you will be dumber for having read them.

Finally, moving away from Taleb’s actually arguments, I have to briefly comment on his writing and tone. The former is, in my opinion, fairly bad and the latter is downright egotistical.

I felt that his writing was scattered and unfocused, which made the book feel somewhat incoherent. And his tone is just insufferable. I mean, enough with the French bashing already. (Ha ha. French people are smelly. Look at me, aren’t I funny?!)

¹ I hope you aren’t upset that I’m writing a bad review of my birthday present. This is not equivalent to saying “I didn’t like my birthday present”. I liked it, as a gift. I wanted to read it and am glad you gave it to me, for no other reason that it has given us something interesting to talk about.

I don’t mean to belabor this topic, so this will be my last post on the subject. Jon pointed to this and it’s a great example of what I was complaining about before.

Namely, it’s Nate Silver leveraging his (inflated) reputation as a Statistical Prediction God to do things (presumably for money) that may be fun for geeky types (myself included) but that do not represent a particularly responsible presentation of the usefulness of statistics to the general public.

So the other night I was in desperate need of something to do other than editing a chapter of my dissertation and so I decided to organize our Taboo cards. I’ll give you a few moment to laugh…

Done? Ok.

It’s not quite as crazy as it sounds; we had just played Taboo with some friends a few nights before, and I had discovered that the cards were all jumbled up: the colors weren’t together and many were upside down.

Anyway. I happened to look at the rules and noticed a discrepancy between the way I’ve always played (I think) and the actual rules. As far as I know, I’ve always played under the assumption that you can say anything once one of your teammates says it, even if that word is taboo. But that’s not in the rules.

Now I can’t recall: did we argue about this once at Inger’s? If we did, I’ve forgotten. I remember some arguments about the parsing of different parts of speech, but not this. If we never discussed it, where did this rule come from?

The friend I had just played with (for the first time) also used this “if your teammates say it so can you” rule. Or at least they didn’t complain.

Does anyone remember the origins of this rule, or any arguments about its validity back in the days of Inger-Delt?

I’ve been lucky enough to be able to increase the amount of soccer playing in my life over the last year or so. Mostly it’s been pick-up and co-ed intramural or city league teams.

I really love soccer, but haven’t always enjoyed actually playing the game, so it’s been great to have fun playing again. I’ve discovered that they key is to find a group of people that are (a) pretty good and (b) not assholes.

(b) is harder than you might think. The main problem is that when it comes to recreational soccer (or athletics in general) many of the assholes aren’t really aware that they are, in fact, assholes. What follows is a simple guide for identifying assholes, even if they happen to be you!

There are essentially three classes of rec soccer assholes.

1. Knowingly malicious assholes
2. Obliviously malicious assholes
3. Momentary assholes

Some basic observations: (1) and (2) are assholes pretty much all the time (when playing soccer). (3) are only assholes when they are (or feel) provoked, resulting in a sudden, usually short lived burst of assholish-ness.

Those falling under (1) generally tend to remain assholes even when not playing soccer, whereas (2) tend to be pretty cool people off the field. (3) will appear like the exact opposite of assholes until suddenly you seem them running at another player wielding a flaming baseball bat and describing unholy acts between themselves and the offending player’s close female relations.

How can you tell if you are one of these assholes? Let’s take them in order of ease of identification:

1. Do people routinely call you an asshole? Are you generally disliked? Do you really enjoy hurting people, both emotionally and physically? You are probably a type (1) asshole. There’s not much we can do about these assholes, except give them a wide berth and if possible ignore them.
2. When playing soccer, do you find yourself routinely exclaiming “But I got the ball!” in order to excuse the obvious pain of someone lying on the ground? Are you often pulled aside by teammates and asked, politely, to chill out? Are you genuinely baffled by both of these trends? You are probably a type (2) asshole. You obviously don’t realize it, but you are a danger to yourself and others. You may not mean any harm, but due either to lack or coordination, excessive enthusiasm or some combination of these, you are placing your limbs in the paths of others at much too high a rate of speed. Luckily, there is hope, as type (2) assholes can be reformed (c.f. Rich). Just concentrate on doing things more slowly, and don’t kick at the ball unless you are really sure you can actually touch the ball. Type (2) assholes should be nurtured, not shunned, for they truly know not what they do. Patient, polite reminders are necessary. Avoid retaliating, as they will not understand why you are fouling them. It will just make them mad, and push them towards type (1).
3. Are you quiet person? Does the thought of actually fighting another human being make you tremble? Do you tend to let things irritate you for so long without saying anything that when you finally lose it, you do things that you regret for years? You are the most elusive asshole: type (3). The key here is to resist that retaliatory urge, perhaps by channeling it elsewhere. Yes, we know that guy fouled you. It’s ok. Buy a punching bag. Strangle a puppy. Something. Type (3) assholes are very difficult to spot. The most reliable method found so far is to place people in a situation where they are playing with a large number of type (2) assholes. The type (3) assholes will show themselves eventually. Once a type (3) assholes has been identified, extreme caution is advised. Pay close attention to their mood and facial expressions. Type (3) assholes hold onto shit for very long periods of time, so their apocalyptic nuclear strike of a foul may come days, or even weeks, after the fact. The key is to learn to recognize the early warning signs of building frustration. The nice part is that simply talking to the type (3) asshole rationally about it often defuses their (subsumed) anger. (Full disclosure: I am a partially reformed type (3) asshole.)

Finally, I should make a note about how we should react to assholes when they do something asshole-ish. We’ve all seen it happen, some type (i) asshole just totally fucking wails on some poor undeserving player. This usually results in large numbers of people running up to the asshole and screaming at his face about what an asshole he is.

Now, I don’t know about you, but I have never, ever, witnessed an asshole (of type (1), (2) or (3)) respond to this by saying, “You know, it never occurred to me, but know that you’re screaming horrible things at me and shoving me, I realize that my actions were wrong. I sincerely apologize and promise to never do that again.” The only thing you are provoking is more asshole-ish behavior. So next time someone’s an asshole on the soccer field, take a few breaths. Relax. Is someone talking to them calmly about their asshole-ish-ness? Yes? Good. No need for you to do anything. Go get a drink of water. Take a breather. Consider yourself lucky that you have now identified an asshole and can avoid them on the field in the future.

So there you have it. If you’ve discovered while reading this that you are, in fact, an asshole, do not despair! There is hope for you! Unless you’re a type (1) asshole, in which case FUCK OFF!

Statistics is changing. All fields of knowledge change, sometimes quickly, sometimes slowly. Statistics has been changing quicker than usual over the past two decades or so.

I think that it is becoming much harder to justify including statistics as a discipline within the field of mathematics. (Others, much more learned than I, have made similar observations .)

Lately, statisticians have become too wrapped up in their identity as mathematicians. This has meant that much of the really exciting work with data being done today is being done by people with CS backgrounds. It’s pretty depressing to read a lot of statistics journals these days. It seems like much of the work being done is filling in theoretical gaps that have relatively little impact on real world data analysis. Many of the details being investigated mathematically assume that a stochastic model is right, and then try to make the estimators better. But I seriously doubt that the biggest limitation of multiple regression analysis in social sciences is really the relative efficiency of one estimator over another. (I’m generalizing; not all research is like this, obviously.)

What makes this worse is that the actual math underlying statistics (beyond the very basics) is just plain boring. There’s a reason that statistics papers put their proofs in the appendix: they aren’t needed to understand the result!

For your edification:

There is strictly speaking no such thing as mathematical proof; we can, in the last analysis, do nothing but point…proofs are what Littlewood and I call gas, rhetorical flourishes designed to affect psychology, pictures on the board in the lecture, devices to stimulate the imagination of pupils. – G.H. Hardy (1928)

I’m certain that Megan McArdle is, in my opinion, far and away the most infuriating blogger in existence.

And yet I’m oddly fascinated by her commentary, which means that I keep checking her blog and watching her on bloggingheads.tv. I’m beginning to feel as though reading her blog is my own version of the book.

Please. Put me out of my misery. Enlighten me! Why am I so powerfully drawn to this train wreck of a blogger like a moth to a flame?

PS – Man, that last sentence sucked. It must be McArdle’s fault.

The recent presidential election provided quite a lot of fodder for those of us that have a fetish for quantitative data. I’m speaking, of course, of the staggering volume of election polling that took place. I have a healthy skepticism for the science (art?) of public opinion polling, but presidential elections seem to be providing ever increasing amounts of public opinion data on a single topic. And if there is anything that I do like, it’s large quantities of data!

Of course, it wasn’t long before we were informed that, really, in order to understand the implications of this glut of polling data, we needed some system for aggregating and combining the information. RealClearPolitics (RCP) was an early leader here. The other heavy hitters are Pollster.com, 538, the lesser known Sam Wang and quite a few others who I will omit for space.

I became a fan very early on of Mark Blumenthal, blogging as MysteryPollster, and then went on to found Pollster.com with political scientist Charles Franklin. In particular, I appreciate a style that emphasizes a restrained approach to data analysis. They are interested in clearly and cleanly displaying data. Nothing more, nothing less. (There’s lots of other commentary on Pollster.com, but this is the heart of what they do.) Most of the other sites, RCP, 538, etc. want to be oracles; Pollster.com wants to be a resource for public information.

In any case, one aspect of (most) sites like these was that they provided aggregated estimates of the level of support for Obama and McCain, not just nationally, but in each state as well. (Sam Wang is an exception here, as are some of the lesser known sites that take a fully Bayesian approach.) Inevitably, the question arises, who’s model did the best? This has been looked at (see here, here, and here ) by others, basically concluding that there wasn’t much of a difference.

The reason this is important is because the complexity of the approaches taken diverged wildly. I won’t go into the gory details, but RCP simply took (unweighted) averages of the most recent polls. Pollster.com fit non-parametric regression curves to all the polling data and 538….well, yeah. Let’s just say that Nate Silver built a lot of machinery up to tackle this problem.

I was curious about how much these three methodologies diverged in accuracy as well (RCP, 538 and Pollster; Chris Bowers didn’t consider RCP, though he did include Gov and Sen races, which I did not). I won’t clog this post with zillions of graphs, but it suffices to say that there were essentially no differences in the accuracy of the predictions made by any of these sites, but that they all did much better than had we simply picked one poll near the election and used that as our prediction. (If this post generates a lot of discussion, I might be convinced to go back and put up some graphs and extended analysis. At the moment, I’m too lazy, so you’ll have to make do with me summarizing my conclusions.)

What can we learn from all this? First, we should always be aware of the diminishing returns of increasing model complexity. Second, we should be frightened by how easily Nate Silver garnered a reputation as being (essentially) infallible by

• Evincing an air of certitude and
• Using methods far more complex than are necessary

Is it really surprising that his background is that of a quant, and not really that of a statistician, as is commonly believed? (Technically, Wiki tells me that his education was in economics, and that he worked as a financial analyst.) This isn’t meant as a harsh knock on Nate Silver, despite appearances. I mean, his model was really accurate. It just wasn’t any better than RCP or Pollster.com.

The rapid rise of Nate Silver worries me, because it sometimes begins to take on a cult-like atmosphere. Nate Silver doesn’t bother me; if I had as much free time as him, I’d probably spend a lot of it doing something similarly too-complex (and fun!). (Indeed, I read the commentary on his site all the time, although Sean Quinn brought way more to that site than Nate Silver ever did, in my opinion.) What frightens me is how easily many people seem to be convinced that “more complicated” is automatically better. Particularly given how well the quants did with their super complicated models for credit scoring in recent years.

PS – I’m being glib about the quants and credit scoring; all I mean is that “we” (someone!) royally screwed that up. I know it’s complicated and don’t mean to single out any particular group for blame.