Monty Hall Problem Pt 1/3: Fallibility of Intuition and the Scratch-&-Win Analogy

The point 1:The Monty Hall Problem” demonstrates that one cannot rely on intuition alone to determine the “truth” of a matter and as such, this should cause one to question their faith in intuition-based ideas.

The point 2: The correct but counter-intuitive answer to “The Monty Hall Problem” (I don’t want to give it away here) is made intuitive via a Scratch-&-Win lottery ticket analogy.

The rant that goes with it:

For those of you who don’t know what what is meant by “The Monty Hall Problem”, I will explain shortly.  First, a short pre-amble on intuitive thinking…

There are those who believe that their instincts provide better insight into the workings of the universe than hard evidence and sound logic, even when the evidence and (often biassed) instincts are in conflict. For example, renowned anti-vaccination advocate and child-killer-by-proxy Jenny McCarthy, who almost single-handedly brought back such popular diseases like Measles and Mumps solely on the basis that her “mommy instincts” outweigh well-established scientific modalities, is one such person. I developed and occasionally teach a course on Information Management that uses the “Monty Hall Problem” (some call it a “paradox”, but I argue that there are no such things as true paradoxes. That is a subject of another post…) to demonstrate that SOME things which appear to be bone-headedly obvious might not be true at all - no matter how much they seem so. Friedrich Nietzsche summed this concept up beautifully when he said:

A casual stroll through the lunatic asylum shows us that faith proves nothing.

On hearing my enthusiasm for this quotation, one may invoke Voltaire, who said, “A witty saying proves nothing”, which does apply to something like, “The best things in life are free”, because one could easily challenge the meaning of “free” or provide examples of favorite things that are not free.  However as witty as Nietzsche’s saying is, it isn’t so shallow as that.  He states no premises that require substanciation.  Regardless of your beliefs or your feelings towards Mr. Nietzsche, this statement stands on its own in that the logic is valid and we all understand the premises (that most if not all of the lunatics are delusional in spite of their personal convictions, a point logically proven if they disagree with each other) to be true – axiomal, if you will.  If you question my statement about the premises, please reply with an explanation why.

I was first impressed with the validity of this sort of logic (and became aware of phenomenological arguments and post-modern thinking…both of which are despicable, but that may be the topic of a future post) when I first saw John Carpenter’s wonderful 1974 sci-fi filmDark Star“, in which the crew of a spaceship attempts to convince an artificially-intelligent bomb not to explode (click to watch video of this scene); the problem being that the bomb-bay doors have failed to open but due to a communication glitch, the bomb is convinced that it has already been dropped and therefore MUST explode in order to “fulfill its destiny”.  To make matters worse, the crew cannot manually stop the countdown…only the sentient bomb has the ability to do this.  Unfortunately, no matter how hard the crew tries, they cannot convince the bomb to stop the countdown because they cannot convince the bomb that it hasn’t been dropped: all of it’s (damaged) sensors says that is is on it’s way to its target.

Acting Captain Doolittle decides that the best strategy is to “teach it phenomenology”, whereby they enlighten the bomb to the idea that our perception of the universe is a product not of actual reality, but the information that our senses provide to us…and that information could be wrong, therefore the bomb should question the order to explode.  Although phenomenology usually doesn’t get you anywhere, Doolittle capitalized on that aspect when he used it to confuse the bomb into questioning its objective to explode:

DOOLITTLE: How do you know you exist?

BOMB #20:   It is intuitively obvious.

DOOLITTLE: Intuition is no proof. What concrete evidence do you have that you exist?

BOMB #20:   Hmmmm… well… I think, therefore I am…my sensory apparatus reveals it to me.

DOOLITTLE: How do you know that the evidence your sensory apparatus reveals to you is correct?  What I’m getting at is this: the only experience that is directly available to you is your sensory data.  And this data is merely a stream of electrical impulses which stimulate your computing center.

BOMB #20:   In other words, all I really know about the outside universe relayed to me through my electrical connections.  Why, that would mean…I really don’t know what the outside universe is like at all, for certain.

Once the bomb understands this idea, it asks the obvious question…

BOMB #20:   True, but since this is so, I have no proof that you are really telling me all this.

…but Doolittle eloquently explains…

DOOLITTLE: That’s all beside the point.  THE CONCEPTS ARE VALID, WHEREVER THEY ORIGINATE!

…and such is the power of internally-consistent logic.

Back to the Nietzsche quotation – his point was basically this: although faith may provide oneself with INTERNAL CERTAINTY (to know…), that does not necessarily translate to EXTERNAL ACCURACY (…but be wrong anyway) as demonstrated by the mutually exclusive, and often disproven, beliefs of the asylum tenants.  Or, as Mark Knopfler of Dire Straits sang in “Industrial Disease”:

Two men say they’re Jesus…ONE of them must be wrong!

But I digress; For the purpose of this post, perhaps we should re-word Nietzsche’s statement as…

A casual look at the history of the Monty Hall Problem shows us that intuition proves nothing.

The challenge of opening people’s eyes to the idea that their cherished instincts can be wrong is that unless you actually gain THEIR acceptance of your position, you have failed. It is one thing to show someone up in a debate in front of an audience and convince the fence-sitters that your idea is supported by the evidence, but it is quite another thing to win the hearts and minds of those who are dug-in to their opposing view (ie – the dogmatic opponent). In fact, the advice usually given to scientists that are about to debate pseudo-scientists – those who abide by fallacious logic and manufactured evidence for a living – is that they shouldn’t even bother trying or they will end up being the one in the insane asylum. But for those up to the challenge, this post provides a means of confronting the power of intuition head-on.

What is “The Monty Hall Problem”?

For those who are unaware of the by-now-very-well-known “Monty Hall Problem” or have seen the movie “21”, I will briefly describe it here. Skip the next three paragraphs to tthe asterix* if you know the problem and solution, even if you don’t agree with the consensus opinion…

In the very real 1970s game show, “Let’s Make a Deal”, the host (actually fellow Canadian Monty Hall – no relation) presents to you three closed doors that are non-descript except for their being numbered 1, 2 & 3.

3 closed doors
You are both aware that behind one door is a valuable prize and behind the other two are little more than booby-prizes. Yes, actual boobies can be an awesome prize, but in this case it’s just an expression for “prize having no value”. You do not know behind which door is the real prize but everyone, including yourself, is aware that Monty, and only Monty, knows which door conceals the real prize. He asks you to pick a door (which you do – ie, #1).

3 closed doors, 1 is selected
As AT LEAST ONE of the remaining doors is certain to have a booby-(ha…boobies!)-prize and Monty knows which one it is, he opens a booby-(joke gets tired fast)-prize door (ie, #3) leaving two closed doors: Yours (#1) and the other one (#2).

3 doors, 2 closed with 1 of those selected and one of the unselected is open
Monty then gives you the opportunity to change your selection (Note: in the show’s history nobody was ever stupid enough to pick the door that Monty had already opened).

The question is this: Is it statistically advantageous to…
(a) stay with the original choice of door #1
(b) switch to unselected door #2 or
(c) do niether (a) nor (b), because it makes no difference?

The most common answer, based on the intuitive conclusion that if it’s either #1 or #2 then there must be an EQUAL chance of the big prize being behind either door, is “(c) – It makes no difference”.

Less common but in my experience still quite common is response “(a) stay with the original choice”, however this response typically comes from people who confuse “statistically advantageous” with “I commit to my decisions”.

Although wrong, the former (c) group tend to be smart people who, like most of us, are simply unaware of the actual statistics at play here (until recently even Nobel-prize-winning mathematicians got it wrong). However, the latter (a) group tend to be dogmatic-minded magical-thinking dumbf**ks that consider their random guesses to somehow transcend logic and evidence.

By elimination (and by statistical proof) the correct answer is “(b) switch to the unselected door”.

3 open doors
*Before you reply to this post with complaints that I am part of the conspiracy to confuse people with mathematical trickery to “prove” something that cannot be true (ala the dastardly Zeno’s Paradoxes such as “Achilles and the Tortoise”, which do this), I say this:

  1. The simplest explanation: When you start out, each door has a 1/3 chance of being a winner.
    3 closed doors, 1 is selected
    When you select a door, the chances yours is a winner is 1/3 and therefore the chances that the prize is behind a door you didn’t select is 2/3.
    3 closed doors, 1 is selected
    Everyone knows that AT LEAST ONE of the two unselected doors is a loser, so when Mr. Hall opens it, he does not change the 1/3 chance that the door you selected is a winner.  Rather, he has merely let you know which of the two unselected doors would have been a bad choice.
    3 closed doors, 1 is selected
    Therefore, the chances that the prize is behind the other unselected door is 2/3 because 1/3+2/3=1, where the “1″ represents the 100% chance that the prize is SOMEWHERE behind the 3 doors.
    3 closed doors, 1 is selected
  2. Google the problem and look at other (mathematical, logical, trials) solutions. If you don’t like them, then it sucks to be you – so get with it…this problem has swung from a sizable amount of of the mathematical community having previously being convinced that it doesn’t matter which door you pick back when the problem was first identified over 20 years ago, to presently the vast majority (there are still some fringe holdouts amongst mathematicians, like those very few ”scientists” that are also creationists – ref “Project Steve“, but these are handy a litmus-tests for incompetence as far as I’m concerned) of generally the SAME COMPETENT PEOPLE convinced that it is statistically advantageous (66% likelihood of winning) if you switch doors, after both doing the relevant math and through real-world test trials required by the last of the hold-outs.
  3. I, too ONCE swore that the answer was (a) and it wasn’t until I had an epiphany* that I came to not accept but understand, INTUITIVELY, that the correct answer is indeed (b). This was a paradigm-shifting moment for me because at that instant, any unfounded faith I may have had in anything (not that I had much to begin with, mind you) immediately evaporated and I became a “born-again” skeptic.
  4. It simply works:  My newfound appreciation of the problem provided me with elevated powers to screw with people (I will get to this later).

*No, that wasn’t her stripper name.  The following is the epiphany I mention in my statement #3 above…

    • Imagine you are at your local convenience store and the proprietor has the only lot of 10 scratch-&-win tickets. Let’s say that he lost his licence to sell State-run lottery tickets and in order to stay in the business he is running his own (fair) lottery.

10 unscratched tickets

    • One of the $10 tickets has a big-money payout ($90, which supposedly guarantees the proprietor a profit of $10, but of course only if he sells them all and continues to do so once someone wins). The remainder are worthless.
    • As he was the creator of this special lottery, it is no secret that the proprietor already knows which of the tickets is good for the prize.
    • You randomly select a ticket from the pile and pay him the $10 for it.

10 unscratched tickets, 1 selected

    • The proprietor, who was stupid enough to lose his licence with the State and also stupid enough to think he will make a profit on the game if he (likely) sells the winning ticket before the 90th sale, offers to make the game “a little more interesting”.
    • He says, “give me $60 and I will increase the chances of your winning the $100 from 10% to 50%. Although the statistics of this is NOT quite in your favour, you were dumb enough to buy a lottery ticket in the first place (or perhaps you are just REALLY curious and want to see where this is going), you accept the challenge and give him $60 for a supposed 50/50 chance to not quite double your money.
    • Knowing that there are AT LEAST 8 losing tickets left in his pile (9, if you picked the winner) as well as which ticket is the actual winner, he scratches 8 of the unselected tickets, leaving one unscratched ticket in your hand and one in front of him on the table. Both of you know that one of these is the winner.

10 tickets, 8 scratched, 1 selected and 1 unselected, each unscratched

    • He says to you, “Now, you can have either the one in your hand OR the one on the table. Which one do you want?” It is now clear that he only and erroneously THOUGHT it was 50/50 odds.
    • However, you remember that when you started the game, the chances of the ticket you originally chose being the winner was only 10% and therefore deduce that the chances of the winning ticket remaining in the pile was 90%.
    • When the proprietor scratched those 8 losing tickets, the original conditions never changed: The chances that the winning ticket remaining on the table continues to be 90%: he simply separated the wheat from the chaff and chances are that he has inadvertently told you where to MOST LIKELY find the winning ticket.
    • You happily swap tickets knowing that there is a 90% chance that your $70 investment returns the $100 prize for a profit of $30.

10 scratched tickets

      • Now, imagine this same game with only three doors…Oops…tickets.

Go have a drink and a think and then come back to read on…

Throughout the entire run of “Let’s Make a Deal”, nobody understood the actual statistics behind the game. It would have only taken one wary statistician to bring this to light in order for the producers to feel the economic pressure to change the rules. It wasn’t until years after it was cancelled that someone figured it out, and she was the object of extreme and unwarranted (but thankfully, not lasting) professional revulsion for years after that.

This post has become absurdly long, so I am breaking it into three parts. Hopefully you now have greater appreciation for not only the elegance and practicality of the correct answer, but also – if the correct answer was a surprise to you – appreciate the implication this has on intuition-based belief sytems, which include faith.

In Part 2 of this series, I will tell the story about how I used the “Monty Hall Problem” to swindle an over-confident employer into sending me to the TAM7 conference in Las Vagas and in Part 3 I will describe my own version, which I call “The JASON Hall Problem” (I’m not an egomaniac, I just couldn’t resist the name), which is an improvement on the original in that it changes the most common intuitive answer from “stay or switch: it doesn’t matter” to “definitely stay”, whilst the correct answer is still to switch.

Comments are welcome and encouraged.  However, before you post, please read my Moderation Policy, which I’ve adopted to control Spam.  Basically, if you link to another website AND you do NOT refer to some specific detail about my post or another commenter’s post, your comment will be trashed before it appears, even if you are kind enough to say only, “I like your blog”.  Sorry ’bout that, but the spam-bots have wrecked it for all of us.

3 thoughts on “Monty Hall Problem Pt 1/3: Fallibility of Intuition and the Scratch-&-Win Analogy

  1. Actually, the original show didn’t quite work this way because Monty didn’t always open another door, and that makes a big difference. For example, if the host ONLY opens another door when you happen to pick the car, then of course it is ALWAYS in your best interest to stay — your chances will be 100%. (You’ll still only win 1/3 of the time total, but the other 2/3 of the time you got a goat and no chance to switch.) On the other hand, if the host always opens a door when you chose a car, plus half the time you chose a goat, then your chances really are 50-50. This is close to what Monty actually did, as far as I know.

    I also disagree that the mathematical community was “95% convinced that it doesn’t matter which door you pick”. If you’re thinking of all the letters to Marilyn vos Savant, remember that that wasn’t a survey of mathematicians but a self-selected group, and someone who disagreed with her would probably be a lot likelier to write than someone who agreed. (Rather like how people on the Internet more often write to disagree than agree. Rather like me…)

    Still, it is pretty disconcerting that so many PhDs would disagree without even bothering to consider why someone might think it was 2/3 for switching. That’s the part that really gets me… if I really think the answer is 1/2 and someone else insists it’s 2/3, then I’m going to be very curious how they reached that conclusion. There’s no obvious cognitive bias that would lead you that way, except maybe the gambler’s fallacy? (Which I’ve seen some supporters of the correct answer be accused of. Funny, because the people who get it wrong guilty of an even weirder sort of retroactive-causation fallacy. “If Monty is about to reveal a goat after I pick, then my odds of picking right the first time are 1/2, not 1/3! But what if after promising to open a door, he gets an urgent call and has to leave before he can do so? Nooo…”)

    • I admit that when I said “95% of mathematicians…” I was thinking of the vos Savant situation but also just being melodramatic, not using any sort of real statistics. Sort of like the saying, “90% of all statistics are made up” :) In that case, my writing was reflective of the way I speak, which is a bad habit because of course the “tell” inflections don’t come across.

      You’ll see that I’ve edited the “95%” text accordingly, and I think the changed text is more interesting than the original.

      To undermine myself even further, even if the letters to vos Savant were 95% negative, that would not be a reliable statistic because people tend to make the effort to write about something only when they have a complaint or need ther thing changed. If you go to a restaurant chain and the waiter is offensive, you complain to the CEO. If he is excellent, you leave a big tip and that’s the end of it, the CEO never hears about that. Unfortunate but true…I’m sure 95% of the time :)

      I was born in 1967, so although I did watch the show during its run, my memories of the game are a bit fuzzy – having been overwhelmed by the costumes the contestants wore (wasn’t that awesome?). I don’t recall his occasional changing of strategy but even if he did, that of course is not the point. The issue is the game as described under the name “The Monty Hall Problem”.

      I completely agree with your curiosity as to why someone would insist on 1/2 when someone else believes so strongly in that which the 1/2-er thinks is absurd. I am fascinated by belief systems and if you read the other 2 parts of this series I think you will see this. This is the reason why I love this problem so much: people’s reactions and their convictions to something that is demonstrably wrong.

      I remember when our kids would argue over a fact (or a toy), whether subjective or objective and especially when one was passionately convinced of his position (or REALLY wanted the toy) and the other was not so committed to his (could live with or without the toy) – and yet they are still fighting…these ARE kids BTW – I would tell the less passionate one, “consider how sure he is of himself, does that not make you question your position at all?” It often worked, moreso as they got older.

      Of course it is just a heuristic negotiation tactic, not a “decider” and it is EXTREMELY important that they realized that just because someone is passionately more convinced of something than you are, that does not mean that they are more (or less) right than you are. In fact, passionate thinkers can be delusional through bias and/or emotional “logic” processes, which often lead to delusion – so I am often suspect of passionate arguments. This is where the MHP comes in: a rare chance to demonstrate through logic and math that even the most passionate belief can be completely wrong.

      Intersting how the thing you ask the 1/2-er to see in the 2/3-er is the very thing that I, a 2/3-er, do see in the 1/2-er.

      I will have a think about your relating to the Gambler’s Fallacy after I’ve had my first cup of coffe this AM.

      BTW – Thanks so much for writing an intelligent and relevant reply to my article. I started posting in January and your’s is the first non-spam comment (unless you are a spammer that puts an inordinate amount of effort into ensuring his post isn’t trashed? – kust kidding! :) I stopped posting a couple of months ago because I figured nobody was reading, so why bother. I have an article on the dangers of celebrating the story of Isaac & Abrahm that has been almost finished for weeks but I had abandoned. I will finish and post it today because of you – thanks again!

      • That’s very gratifying to hear, thanks!

        (By the way, do want to LOWER YOUR INTEREST RATES?)

        I myself have a never-read blog which I almost never update, but I just recently wrote a bit about Monty Hall simply because I’ve been thinking about it lately and came up with a whole new angle I’d love to see some naysayer try to refute. (Although the naysayers are probably a dying breed at this point…)

Leave a Reply

Your email address will not be published.

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>