The Hanover Collaboration

A few of my friends are going through residency matching with hospitals right now and by coincidence we ended up discussing that very problem in an optimization class.

The problem, as posed, is to find a perfect stable matching. The simple explanation is they are trying to find any matching such that there is no pair of assignments in which the two hospitals and two students would (all four) prefer to switch. This has little to do with optimality, except for that if anything were changed, at least one player would be worse off. It is notable that the classic version of the algorithm would produce a matching that favored one group or the other. In the residency problem, I wonder which is given the preference?...

The main point of this post is just to point interested people to that Wikipedia article. Beyond that, more recent scholarship has identified a number of nice properties of the problem (e.g. there exists a self-dual linear relaxation), which make it easier than it looks to find feasible assignments. As a result, I’m suddenly quite suspicious that DOC trips could effectively use commercial optimization software for trip assignments. That would be fun—perhaps even more fun than doing it by hand. Remember that, guys?

Seeing how it is fast becoming job interview season, I thought I’d share a parlor trick with you guys. Business-type people are very easily impressed if you can do compounding interest in your head.
It boils down to being able to estimate natural logarithms. Here’s how:

Here’s an applet that lets you play rock/paper/scissors against the computer. Some of you might find the following two experiments interesting. You should do the experiments before reading how the algorithms work.

1. Play with the window scaled such that you can’t see the results. In other words, do your best to create a random sequence. Check in every 25 clicks or so.
2. Watch the computer and see how well you can “fake it out.”

I don’t mean to belabor this topic, so this will be my last post on the subject. Jon pointed to this and it’s a great example of what I was complaining about before.

Namely, it’s Nate Silver leveraging his (inflated) reputation as a Statistical Prediction God to do things (presumably for money) that may be fun for geeky types (myself included) but that do not represent a particularly responsible presentation of the usefulness of statistics to the general public.

Statistics is changing. All fields of knowledge change, sometimes quickly, sometimes slowly. Statistics has been changing quicker than usual over the past two decades or so.

I think that it is becoming much harder to justify including statistics as a discipline within the field of mathematics. (Others, much more learned than I, have made similar observations .)

Lately, statisticians have become too wrapped up in their identity as mathematicians. This has meant that much of the really exciting work with data being done today is being done by people with CS backgrounds. It’s pretty depressing to read a lot of statistics journals these days. It seems like much of the work being done is filling in theoretical gaps that have relatively little impact on real world data analysis. Many of the details being investigated mathematically assume that a stochastic model is right, and then try to make the estimators better. But I seriously doubt that the biggest limitation of multiple regression analysis in social sciences is really the relative efficiency of one estimator over another. (I’m generalizing; not all research is like this, obviously.)

What makes this worse is that the actual math underlying statistics (beyond the very basics) is just plain boring. There’s a reason that statistics papers put their proofs in the appendix: they aren’t needed to understand the result!

For your edification:

There is strictly speaking no such thing as mathematical proof; we can, in the last analysis, do nothing but point…proofs are what Littlewood and I call gas, rhetorical flourishes designed to affect psychology, pictures on the board in the lecture, devices to stimulate the imagination of pupils. – G.H. Hardy (1928)