The point: Change the option of the “Monty Hall Problem” from “switch one door for another” to “switch TWO doors for one door” in order to eliminate the “preference bias noise” of the original version of the problem and make the correct solution even more counter-intuitive.

The rant that goes with it:

In Parts 1 & 2 of this 3-part series, I demonstrated how “Monty Hall Problem” can be intuitively explained, the implication that it has on the reliability of intuition and how such faith in one’s false convictions can expose you to predators…like me. In this Part 3, I propose an alternative version of the problem that I cheekily call the “Jason Hall Problem” (there are those, including my wife, who will tell you that this is merely one in a long list…).

In the “Monty Hall Problem”, the premise of trading one door for one other door often confuses people into thinking the problem is about preference as opposed to statistics. In other words, “If it doesn’t matter whether I stay or switch, staying is better because I stick to my commitments”. Even though these people believe that statistically it doesn’t make a difference, when asked what is the best thing to do they insist that it is either “better” to keep their original choice (if they tilt towards the psychotic) or to switch doors (if they tilt towards the neurotic). The commitment to one or the other over principal alone is very strong, a fact made clear in this anecdote:

While doing the 10-card demonstration (ref Part 2, the trials with my boss) for one of my colleagues, he insisted repeatedly that it was in his best interest to stay with his original choice, even as I continued to explain to him that the actual statistics gave him a 90% chance of winning if he switched. Then I switched tactics: I put $5 on the table and told him that the money was his if he ended up with the winning card. At that point, he said it was obvious that he should switch, which he did. I then asked him, before telling him that he had won the $5, “and if there were no money involved?” His response, amazingly, was, “then I wouldn’t switch”. I concluded from this experiment that some people are really, really messed up beyond saving…and that preference bias is strong, even can be bought-off.

Changing the problem so that the correct answer is to trade away one number of doors in exchange for a different number of doors eliminates this “preference noise” of the original problem. Furthermore, to emphasize counter-intuitive properties of this problem in the revised version, the correct answer must (of course) be one where the player must give away more doors than he gets back in order to have the best statistical advantage. As impossible as this seems, if you truly understand the original version of the problem, it is not so difficult to achieve, as I will explain.

In my version, there are 5 doors rather than 3.

Mr. Hall (that would me ME this time around) asks the contestant to select any 2 doors from the 5…

…and then he (I) open 2 of the unselected doors with booby-prizes (hee hee, that’s funny again).  All other parameters of the problem are the same as the original “Monty” 3-door version.

At this point in the 5-door version, the contestant has selected closed doors #1 & #2, the host (that’s me again) has opened doors #4 & #5 to reveal crap prizes and the unclaimed #3 door remains closed.

The question is this: Will the contestant keep BOTH of the doors he picked OR will he trade BOTH of his doors for the ONE closed door that he did not pick? What would YOU do? (please just PRETEND that you don’t know that the problem is engineered so that the least-intuitive answer is the correct one!).

The answer:  It is in your best interest to switch (of course)

Although the intuitive response of someone unfamiliar with the statistics involved would most likely choose to keep his two doors on the basis that he apparantly has a 2/3 chance of winning, the corrrect answer is to switch his two doors for the one unselected door because in fact he would be trading a 2/5 chance for a slightly better 3/5 chance of winning, as shown in this “replay” where each guitar pick represents a 1/5 chance of the door hiding the prize:

One in the bush is worth (more than) two in the hand!

Comments are welcome and encouraged.  However, before you post, please read my Moderation Policy, which I’ve adopted to control Spam.  Basically, if you link to another website AND you do NOT refer to some specific detail about my post or another commenter’s post, your comment will be trashed before it appears, even if you are kind enough to say only, “I like your blog”.  Sorry ’bout that, but the spam-bots have wrecked it for all of us.

The point: When it comes to matters-of-fact, logic/evidence-based positions are superior to intuition-based positions and this can be exploited for profit.

The rant that goes with it:

In Part 1 of this 3-part series, we saw how learning to intuitively appreciate the “Monty Hall Problem” demonstrates that intuition alone is not a reliable means of determining the way things actually work. In this part, I anecdotally will show how this knowledge can be used to really screw people over. Remember: with such great power comes great responsibility, not to mention the chance of getting your teeth smashed-in, so wield this weapon with respect and care!

The beauty of the “Monty Hall Problem” is that although it is just as easy to come to the wrong conclusion as it is difficult to accept the correct conclusion, the right answer can be proven, in court if necessary, making it a perfect tool for a con artist. The idea is simple: If the wrong answer is so intuitive that someone will remain committed to that wrong answer, even AFTER you’ve TOLD them the correct answer, the opportunity to convince them to place a high-stakes wager against the answer that you can ultimately PROVE to be correct is very difficult to resist to any self-respecting swindler. It is the timeless story of using someone’s greed against them. Case and point:

In 2009 while I was working under an expat contract in Qatar, I aspired to attend The 7^th Amazing Meeting (TAM7) conference on critical-thinking in Las Vegas with my son Adrian. Unfortunately, I was unable to go because the project was at a critical stage and the Area Manager refused to grant any leave during that period. This was doubly-unfortunate because I had chosen to split my one annual business-class ticket entitlement into two economies, one of which I had already used to go to Canada (for far less than the value of ½ a business-class) and the right to use balance towards the value of second ticket was about to expire. In short, I wanted to go ½-way around the world to Las Vegas and if only the ADM would sign-off, I would get there and back for free.

Like a kid repeatedly asking his dad for a cookie before dinner, I asked him three times to authorize the trip and like the prick that my dad could be, he continually refused to provide a cookie…er…his signature. Finally, he asked me, “What’s this conference all about anyway?” I told him, “it’s about skepticism and critical thinking”, to which he replied, “what’s critical thinking?”

I explained to him that critical thinking is all about the application of high standards of introspective analysis when forming or considering ideas and that this includes the very challenging tasks of filtering out personal bias, recognizing fallacious logic, questioning dubious premises, learning to accept evidence-based conclusions even if they are inconvenient to you, etc. Considering his perplexed look, I decided to give him an example. I described the “Monty Hall Problem” to him and told him that the techniques of critical thinking help to cut through the intellectual noise that prevent one from not only coming to, but appreciating the correct solution.

When I gave him the question of whether to switch, stay or tell me it doesn’t matter, he came to the intuitive-based conclusion “it doesn’t matter whether I stay or switch” and then scoffed at the correct answer of “best to switch” when I gave it to him. Both of these are common responses, however, even as I explained the logic behind the reason for switching doors, he continued to stand by his original conclusion, becoming very distressed in the process. He apparently had trouble dealing with someone so deluded by such pseudo-math and misguided thinking.

Seeing an opportunity, I proposed, “If you are so sure that I’m wrong about this, then ‘Let’s Make a Deal’, shall we? If I can get you to admit that your answer is wrong and mine is right in less than 10 minutes, you sign-off on our trip to Las Vegas. If I can’t do this in 10 minutes then I will stop asking for your signature and you’ll never hear of this again.”

“You’re on!” he exclaimed, certain that he couldn’t possibly lose.

In the first 2 minutes, I explained the 10-ticket Scratch-&-Win analogy to him, but he wasn’t impressed (I said that it was a good teaching device; I never said it worked EVERY time).

Having failed at the theoretical, I went for the practical in the form of repeated trials: I dug through my desk (typically a pig-sty, iso that cost me another 2 minutes) and found a deck of cards. I took out 9 red-suit cards and one black-suit card to represent losing and winning tickets respectively, shuffled them and put the 10 cards face-down in a row front of him such that I knew where the black suit was. I had him select a card and then I turned up 8 red cards, leaving his card and the 10^th face down on the table. I then asked him, “stay or switch?” True to his belief, he said, “It doesn’t matter”. When he turned over his card, it was a loser. Of course, we know there was a 90% chance that it lose, no surprise there.

Yes, he got it eventually, and fully 2 minutes before my time was up. He signed and I sent him a postcard from Las Vegas.  TAM was awesome, by the way.

I have since been tempted to run this sure bet for cash, but I just can’t bring myself to do it because I have a terrible poker-face and I just can’t bring myself to take advantage of people in such a way (…very often). Perhaps you are not so principled as I am. Good luck with that.

In the final Part 3 of this series, I propose an alternate version of the problem that both eliminated the 50/50 conundrum with the added benifit of being more counter-intuitive than the original.

Comments are welcome and encouraged.  However, before you post, please read my Moderation Policy, which I’ve adopted to control Spam.  Basically, if you link to another website AND you do NOT refer to some specific detail about my post or another commenter’s post, your comment will be trashed before it appears, even if you are kind enough to say only, “I like your blog”.  Sorry ’bout that, but the spam-bots have wrecked it for all of us.

The point 1: “The Monty Hall Problem” demonstrates that one cannot rely on intuition alone to determine the “truth” of a matter and as such, this should cause one to question their faith in intuition-based ideas.

The point 2: The correct but counter-intuitive answer to “The Monty Hall Problem” (I don’t want to give it away here) is made intuitive via a Scratch-&-Win lottery ticket analogy.

The rant that goes with it:

For those of you who don’t know what what is meant by “The Monty Hall Problem”, I will explain shortly.  First, a short pre-amble on intuitive thinking…

There are those who believe that their instincts provide better insight into the workings of the universe than hard evidence and sound logic, even when the evidence and (often biassed) instincts are in conflict. For example, renowned anti-vaccination advocate and child-killer-by-proxy Jenny McCarthy, who almost single-handedly brought back such popular diseases like Measles and Mumps solely on the basis that her “mommy instincts” outweigh well-established scientific modalities, is one such person. I developed and occasionally teach a course on Information Management that uses the “Monty Hall Problem” (some call it a “paradox”, but I argue that there are no such things as true paradoxes. That is a subject of another post…) to demonstrate that SOME things which appear to be bone-headedly obvious might not be true at all - no matter how much they seem so. Friedrich Nietzsche summed this concept up beautifully when he said:

A casual stroll through the lunatic asylum shows us that faith proves nothing.

On hearing my enthusiasm for this quotation, one may invoke Voltaire, who said, “A witty saying proves nothing”, which does apply to something like, “The best things in life are free”, because one could easily challenge the meaning of “free” or provide examples of favorite things that are not free.  However as witty as Nietzsche’s saying is, it isn’t so shallow as that.  He states no premises that require substanciation.  Regardless of your beliefs or your feelings towards Mr. Nietzsche, this statement stands on its own in that the logic is valid and we all understand the premises (that most if not all of the lunatics are delusional in spite of their personal convictions, a point logically proven if they disagree with each other) to be true – axiomal, if you will.  If you question my statement about the premises, please reply with an explanation why.

I was first impressed with the validity of this sort of logic (and became aware of phenomenological arguments and post-modern thinking…both of which are despicable, but that may be the topic of a future post) when I first saw John Carpenter’s wonderful 1974 sci-fi film “Dark Star“, in which the crew of a spaceship attempts to convince an artificially-intelligent bomb not to explode (click to watch video of this scene); the problem being that the bomb-bay doors have failed to open but due to a communication glitch, the bomb is convinced that it has already been dropped and therefore MUST explode in order to “fulfill its destiny”.  To make matters worse, the crew cannot manually stop the countdown…only the sentient bomb has the ability to do this.  Unfortunately, no matter how hard the crew tries, they cannot convince the bomb to stop the countdown because they cannot convince the bomb that it hasn’t been dropped: all of it’s (damaged) sensors says that is is on it’s way to its target.

Acting Captain Doolittle decides that the best strategy is to “teach it phenomenology”, whereby they enlighten the bomb to the idea that our perception of the universe is a product not of actual reality, but the information that our senses provide to us…and that information could be wrong, therefore the bomb should question the order to explode.  Although phenomenology usually doesn’t get you anywhere, Doolittle capitalized on that aspect when he used it to confuse the bomb into questioning its objective to explode:

DOOLITTLE: How do you know you exist?

BOMB #20:   It is intuitively obvious.

DOOLITTLE: Intuition is no proof. What concrete evidence do you have that you exist?

BOMB #20:   Hmmmm… well… I think, therefore I am…my sensory apparatus reveals it to me.

DOOLITTLE: How do you know that the evidence your sensory apparatus reveals to you is correct?  What I’m getting at is this: the only experience that is directly available to you is your sensory data.  And this data is merely a stream of electrical impulses which stimulate your computing center.

BOMB #20:   In other words, all I really know about the outside universe relayed to me through my electrical connections.  Why, that would mean…I really don’t know what the outside universe is like at all, for certain.

Once the bomb understands this idea, it asks the obvious question…

BOMB #20:   True, but since this is so, I have no proof that you are really telling me all this.

…but Doolittle eloquently explains…

DOOLITTLE: That’s all beside the point.  THE CONCEPTS ARE VALID, WHEREVER THEY ORIGINATE!

…and such is the power of internally-consistent logic.

Back to the Nietzsche quotation – his point was basically this: although faith may provide oneself with INTERNAL CERTAINTY (to know…), that does not necessarily translate to EXTERNAL ACCURACY (…but be wrong anyway) as demonstrated by the mutually exclusive, and often disproven, beliefs of the asylum tenants.  Or, as Mark Knopfler of Dire Straits sang in “Industrial Disease”:

Two men say they’re Jesus…ONE of them must be wrong!

But I digress; For the purpose of this post, perhaps we should re-word Nietzsche’s statement as…

A casual look at the history of the Monty Hall Problem shows us that intuition proves nothing.

The challenge of opening people’s eyes to the idea that their cherished instincts can be wrong is that unless you actually gain THEIR acceptance of your position, you have failed. It is one thing to show someone up in a debate in front of an audience and convince the fence-sitters that your idea is supported by the evidence, but it is quite another thing to win the hearts and minds of those who are dug-in to their opposing view (ie – the dogmatic opponent). In fact, the advice usually given to scientists that are about to debate pseudo-scientists – those who abide by fallacious logic and manufactured evidence for a living – is that they shouldn’t even bother trying or they will end up being the one in the insane asylum. But for those up to the challenge, this post provides a means of confronting the power of intuition head-on.

What is “The Monty Hall Problem”?

For those who are unaware of the by-now-very-well-known “Monty Hall Problem” or have seen the movie “21”, I will briefly describe it here. Skip the next three paragraphs to tthe asterix* if you know the problem and solution, even if you don’t agree with the consensus opinion…

In the very real 1970s game show, “Let’s Make a Deal”, the host (actually fellow Canadian Monty Hall – no relation) presents to you three closed doors that are non-descript except for their being numbered 1, 2 & 3.

The question is this: Is it statistically advantageous to…
(a) stay with the original choice of door #1
(b) switch to unselected door #2 or
(c) do niether (a) nor (b), because it makes no difference?

The most common answer, based on the intuitive conclusion that if it’s either #1 or #2 then there must be an EQUAL chance of the big prize being behind either door, is “(c) – It makes no difference”.

Less common but in my experience still quite common is response “(a) stay with the original choice”, however this response typically comes from people who confuse “statistically advantageous” with “I commit to my decisions”.

Although wrong, the former (c) group tend to be smart people who, like most of us, are simply unaware of the actual statistics at play here (until recently even Nobel-prize-winning mathematicians got it wrong). However, the latter (a) group tend to be dogmatic-minded magical-thinking dumbf**ks that consider their random guesses to somehow transcend logic and evidence.

By elimination (and by statistical proof) the correct answer is “(b) switch to the unselected door”.

1. The simplest explanation: When you start out, each door has a 1/3 chance of being a winner.When you select a door, the chances yours is a winner is 1/3 and therefore the chances that the prize is behind a door you didn’t select is 2/3.Everyone knows that AT LEAST ONE of the two unselected doors is a loser, so when Mr. Hall opens it, he does not change the 1/3 chance that the door you selected is a winner.  Rather, he has merely let you know which of the two unselected doors would have been a bad choice.Therefore, the chances that the prize is behind the other unselected door is 2/3 because 1/3+2/3=1, where the “1″ represents the 100% chance that the prize is SOMEWHERE behind the 3 doors.
2. Google the problem and look at other (mathematical, logical, trials) solutions. If you don’t like them, then it sucks to be you – so get with it…this problem has swung from a sizable amount of of the mathematical community having previously being convinced that it doesn’t matter which door you pick back when the problem was first identified over 20 years ago, to presently the vast majority (there are still some fringe holdouts amongst mathematicians, like those very few ”scientists” that are also creationists – ref “Project Steve“, but these are handy a litmus-tests for incompetence as far as I’m concerned) of generally the SAME COMPETENT PEOPLE convinced that it is statistically advantageous (66% likelihood of winning) if you switch doors, after both doing the relevant math and through real-world test trials required by the last of the hold-outs.
3. I, too ONCE swore that the answer was (a) and it wasn’t until I had an epiphany* that I came to not accept but understand, INTUITIVELY, that the correct answer is indeed (b). This was a paradigm-shifting moment for me because at that instant, any unfounded faith I may have had in anything (not that I had much to begin with, mind you) immediately evaporated and I became a “born-again” skeptic.
4. It simply works:  My newfound appreciation of the problem provided me with elevated powers to screw with people (I will get to this later).

*No, that wasn’t her stripper name.  The following is the epiphany I mention in my statement #3 above…

• Imagine you are at your local convenience store and the proprietor has the only lot of 10 scratch-&-win tickets. Let’s say that he lost his licence to sell State-run lottery tickets and in order to stay in the business he is running his own (fair) lottery.

• One of the $10 tickets has a big-money payout ($90, which supposedly guarantees the proprietor a profit of $10, but of course only if he sells them all and continues to do so once someone wins). The remainder are worthless.
• As he was the creator of this special lottery, it is no secret that the proprietor already knows which of the tickets is good for the prize.
• You randomly select a ticket from the pile and pay him the $10 for it.

• The proprietor, who was stupid enough to lose his licence with the State and also stupid enough to think he will make a profit on the game if he (likely) sells the winning ticket before the 90^th sale, offers to make the game “a little more interesting”.
• He says, “give me $60 and I will increase the chances of your winning the $100 from 10% to 50%. Although the statistics of this is NOT quite in your favour, you were dumb enough to buy a lottery ticket in the first place (or perhaps you are just REALLY curious and want to see where this is going), you accept the challenge and give him $60 for a supposed 50/50 chance to not quite double your money.
• Knowing that there are AT LEAST 8 losing tickets left in his pile (9, if you picked the winner) as well as which ticket is the actual winner, he scratches 8 of the unselected tickets, leaving one unscratched ticket in your hand and one in front of him on the table. Both of you know that one of these is the winner.

• He says to you, “Now, you can have either the one in your hand OR the one on the table. Which one do you want?” It is now clear that he only and erroneously THOUGHT it was 50/50 odds.
• However, you remember that when you started the game, the chances of the ticket you originally chose being the winner was only 10% and therefore deduce that the chances of the winning ticket remaining in the pile was 90%.
• When the proprietor scratched those 8 losing tickets, the original conditions never changed: The chances that the winning ticket remaining on the table continues to be 90%: he simply separated the wheat from the chaff and chances are that he has inadvertently told you where to MOST LIKELY find the winning ticket.
• You happily swap tickets knowing that there is a 90% chance that your $70 investment returns the $100 prize for a profit of $30.

• Now, imagine this same game with only three doors…Oops…tickets.

Go have a drink and a think and then come back to read on…

Throughout the entire run of “Let’s Make a Deal”, nobody understood the actual statistics behind the game. It would have only taken one wary statistician to bring this to light in order for the producers to feel the economic pressure to change the rules. It wasn’t until years after it was cancelled that someone figured it out, and she was the object of extreme and unwarranted (but thankfully, not lasting) professional revulsion for years after that.

This post has become absurdly long, so I am breaking it into three parts. Hopefully you now have greater appreciation for not only the elegance and practicality of the correct answer, but also – if the correct answer was a surprise to you – appreciate the implication this has on intuition-based belief sytems, which include faith.

In Part 2 of this series, I will tell the story about how I used the “Monty Hall Problem” to swindle an over-confident employer into sending me to the TAM7 conference in Las Vagas and in Part 3 I will describe my own version, which I call “The JASON Hall Problem” (I’m not an egomaniac, I just couldn’t resist the name), which is an improvement on the original in that it changes the most common intuitive answer from “stay or switch: it doesn’t matter” to “definitely stay”, whilst the correct answer is still to switch.

Comments are welcome and encouraged.  However, before you post, please read my Moderation Policy, which I’ve adopted to control Spam.  Basically, if you link to another website AND you do NOT refer to some specific detail about my post or another commenter’s post, your comment will be trashed before it appears, even if you are kind enough to say only, “I like your blog”.  Sorry ’bout that, but the spam-bots have wrecked it for all of us.