The Hanover Collaboration

I think I’ve got Netflix pretty well trained to give me good movies. Of course this is largely a good job on their part; but I think that I had something to do with it.

I only rate 5s and 1s. Everything in between I don’t rate.

I’m pretty sure that Netflix still uses quadratic loss when predicting your ratings. I think this is really holding them back since it puts an artificial central pressure on ratings. I’m also suspicious that there is a thresholding effect at 4 stars. Specifically, for clustering purposes, I think considers 4 and 5 stars to be the same thing, “movies you liked”. It finds other people who have a high intersection of 4—5 movies and uses them as predictors.

Early on, I rated X-men 3 at the 4 level which led to systematic over-prediction of franchise action movies. That was a mistake. I fixed that by downgrading it to 3 but then putting a 5 on X-Men 1.

Every time I see a bad movie, I make sure to tell Netflix, especially if everyone else rated it highly, e.g., Million Dollar Baby.

If your wireless usage is between 550 and 2100 min/mo and you’re on AT&T, it might strike you as a good idea to get the the 2100 min/mo plan to take advantage of rollover minutes. That is because you don’t have to worry about going negative and paying the ridiculous premium, while at the same time taking advantage on the price non-linearity.

Dahndest thing happened to me today. I gave my usual class 4,5 class to a bunch of kids at MIT. Not only that, it was in that crazy fahking Statar Center up in this fancy-pants conference room.

They were all like 25-30 yeah olds so I assume theya all PhD students oha post-docs. So I ask them, “Whya you all wantin’ to drive fahklifts.” They look at me funny and one says, “because we’ve got one.” Fahkin’ MIT kids!

Looking at the Michigan case again through the lense of Ricci I think I’ve decided how to do affirmative action.

This riddle is called The Unexpected Hanging Paradox

A judge tells a condemned prisoner that he will be hanged at noon on one weekday in the following week but that the execution will be a surprise to the prisoner. He will not know the day of the hanging until the executioner knocks on his cell door at noon that day. Having reflected on his sentence, the prisoner draws the conclusion that he will escape from the hanging. His reasoning is in several parts. He begins by concluding that the “surprise hanging” can’t be on a Friday, as if he hasn’t been hanged by Thursday, there is only one day left – and so it won’t be a surprise if he’s hanged on a Friday. Since the judge’s sentence stipulated that the hanging would be a surprise to him, he concludes it cannot occur on Friday. He then reasons that the surprise hanging cannot be on Thursday either, because Friday has already been eliminated and if he hasn’t been hanged by Wednesday night, the hanging must occur on Thursday, making a Thursday hanging not a surprise either. By similar reasoning he concludes that the hanging can also not occur on Wednesday, Tuesday or Monday. Joyfully he retires to his cell confident that the hanging will not occur at all. The next week, the executioner knocks on the prisoner’s door at noon on Wednesday — which, despite all the above, will still be an utter surprise to him. Everything the judge said has come true.

What is wrong? Is anybody abusing logic here? If so, who?

Suppose that he was told the execution would be either Monday or Tuesday (equivalently, the week only has two days) and he was executed on Monday. Is this case different?

A while back Christopher Hitchens issues an open invitation to debate atheism with anybody. One of these debates turned into a movie that comes out (straight to DVD!) tomorrow. Honestly, I’m pretty confident that the movie is going to be consummately boring not least because Wilson concedes a lot and seems primarily interested in the “Without religion, i.e., if life is meaningless, how do you establish morality?” question. That should be a pretty easy one.

Anyway, I’m just posting to tell people to listen to the last little bit of this Laura Ingraham appearance where, on the buzzer, Hitch nails a three-pointer from half-court.

A few of my friends are going through residency matching with hospitals right now and by coincidence we ended up discussing that very problem in an optimization class.

The problem, as posed, is to find a perfect stable matching. The simple explanation is they are trying to find any matching such that there is no pair of assignments in which the two hospitals and two students would (all four) prefer to switch. This has little to do with optimality, except for that if anything were changed, at least one player would be worse off. It is notable that the classic version of the algorithm would produce a matching that favored one group or the other. In the residency problem, I wonder which is given the preference?...

The main point of this post is just to point interested people to that Wikipedia article. Beyond that, more recent scholarship has identified a number of nice properties of the problem (e.g. there exists a self-dual linear relaxation), which make it easier than it looks to find feasible assignments. As a result, I’m suddenly quite suspicious that DOC trips could effectively use commercial optimization software for trip assignments. That would be fun—perhaps even more fun than doing it by hand. Remember that, guys?

Saletan just posted an article about gender selection that reminded me a lot one I wrote.

In retrospect, my position was insufficiently nuanced to completely answer Bill’s question, so I’d like to clarify it.

I think that all the stated techniques are “okay” which should be interpreted as “should be left legally permisable.” From the context it should be clear that there is an ordering of preference going increasing from later abortions up to special condom. I guess you could say that this preference is “moral”. I like to think though that my system of ethics is more straightforward than that though.

So let me briefly write out Tom’s system of Ethics, (TSE).
Assume there is some global function (subject to restrictions that are beyond the scope of this article) that assigns values to various outcomes. An action’s “goodness” would simply the expected change of this global function. However, we cannot globally agree on this function.

Assume that we can somehow establish “robust consensus” on some convex family of functions. Any action with expectation greater or equal to zero for some function in the interior of this family is “permisable.” We can define a partial order such that an action is “better” if and only if there does not exist a function in the family which contradicts this preference. But this is not a full order, and certain preferences will be left to personal measurement.

We could try to define universal goodness in terms of measures over this family, but that’s what I would call “hide the potato”—agreeing on measures will be no easier than agreeing on a global objective.

Seeing how it is fast becoming job interview season, I thought I’d share a parlor trick with you guys. Business-type people are very easily impressed if you can do compounding interest in your head.
It boils down to being able to estimate natural logarithms. Here’s how:

President Obama wants to give a speech to the nations school students next Tuesday at noon. The country is so polarized that, apearantly, this is a big controversy. Georgetown, (MA), for instance will not be showing it.

Suppose that the President were Republican and the angry parents were Democrats, how do you think this would play out?

1) It wouldn’t. Democrats would suck it up as “civic duty.”
2) It wouldn’t. Democrats don’t have a good enough network (mainly radio shows) to rally enough angry phone calls to schools.
3) They’d do the same, but then get hung out to dry for a “lack of respect.”
4) It would play out just the same.

I’m thinking (1), because I really, truly, hated Bush, but if he were going to give the kids a pep-talk on the first day of school, I’d think that was fine, good in fact. The president is important; they might listen.

Everybody who is hand-wringing about how “political” the speech might be is not thinking about it very hard. When Obama speaks, he is nothing if not cautious. If he said anything even remotely political, even remotely assailable, he would (deservedly?) suffer for it. Those kids are guaranteed some of the A-grade flowery bullshit. But maybe it will inspire a kid or two to work harder, stay in school or become better citizens. Whether or not it’s worth the time is another argument, but I think it probably is.

Regarding the other complaint regarding “indoctrination”, let’s be honest, it’s there and it’s pervasive, but it’s not coming from Obama.

Somebody at work asked about the chicken scratchings I used in the snack room and in response to my solution said something fawning about how neat it was that I could just change my number representation however it suited me.

As Jon would guess, that led to a conversation about balanced tertiary, and the following problem arose.

You have a balance with two pans and an object with integer mass, N, that you would like to determine. You have the following known masses:
let k be an odd, positive integer. You have (_k_ – 1)/2 of each mass of weight kⁿ for n in the non-negative integers. With this set of masses, each integer will have a unique representation.

You would like to determine N in a minimum number of weighings. Any time you add or remove a single mass from the scale counts as a weighing.

For instance let k=3, and I wanted to put 128g (=243-81-27-9+3-1) on the scale when 256g (=243+9+3+1) was on their previously. To do so counts as 6 weighings.

I’m looking for the big-Oh (in terms of N,k) of your strategy. From the above example I think it would be easy to argue that binary search is O(log ² (N)). Can anyone do better?

At work we’ve got a snack room and there is a tab on the wall. Attached to it is a pen rather than a pencil as you would expect in a perfect world… Anyway, I’m struck by how inefficiently most people use the space in it, forcing it to be pages and pages long.

All the prices are divisible by five and go from 5c up to $1 with 45, 55, 65, and 95cents all absent. Come up with a scheme for keeping track of the cumulative sum of transactions that doesn’t use a lot of space.

Benchmark: You’ve got enough room for about 80 characters, so you should be able to get at least 160 items before starting another page.

Hard version: You maintain a fixed number strings of {0,1}*. After each transaction (containing potentially multiple items) you may add as many characters to whatever strings you’d like. From these strings you need to be able to construct the sum of the transactions.

You can assume each transaction comes from a finite set with a known probability distribution. Come up with a scheme that minimizes the expected bits per transaction. You’re welcome to make limiting case arguments, but winners are going to be picked at 640 bits.

Generally, I think “talking tough” is bad foreign policy. Keep in mind that in every country you’ll find people who agree with you and people who don’t. Our “tough talk” tends to buttress the people who disagree with you. Apropos of Iranian elections, John Dickerson basically agrees.

Fred Kaplan makes the argument that now is a moment when talking tough to the Iranian government wouldn’t be construed as talking tough to the Iranian people. As a result, it wouldn’t necessarily be counterproductive.

If you were the President, what would you say?

I’m getting the impression that the standard for educational TV show has really dropped since we were young. I regularly watch National Geographic specials, but they are dominated by “dramatic re-enactments” with really minimal and meaningless narration. I was thinking that the difference is that I’m pickier now—that Nova wasn’t that much better. On the other hand, I’ve always been suspicious that Nova couldn’t have done that kind of thing, even if it wanted to.

I’m in the middle of watching Scientific American Frontiers after watching Nova, and I can assure you, they are much better than anything else on the TV currently.

What I don’t understand is why isn’t (e.g.) the National Geographic Channel comparable? It isn’t like Nova is very expensive to produce. Is it that it is impossible to sell commercials during shows this?

This ad really resonates with me—it makes me want to work for intel. I’m not sure why. Probably because it is a spot-on potrayal about what geeky really means. Maybe its because the usual portrayal of geeky is so grating (e.g. hire geeks to set up your DVR!).

If Bertsimas walked into my lab, I swear to God, it would be just like that. I bet I could make money selling the t-shirt.