The point 1: “The Monty Hall Problem” demonstrates that one cannot rely on intuition alone to determine the “truth” of a matter and as such, this should cause one to question their faith in intuition-based ideas.
The point 2: The correct but counter-intuitive answer to “The Monty Hall Problem” (I don’t want to give it away here) is made intuitive via a Scratch-&-Win lottery ticket analogy.
The rant that goes with it:
For those of you who don’t know what what is meant by “The Monty Hall Problem”, I will explain shortly. First, a short pre-amble on intuitive thinking…
There are those who believe that their instincts provide better insight into the workings of the universe than hard evidence and sound logic, even when the evidence and (often biassed) instincts are in conflict. For example, renowned anti-vaccination advocate and child-killer-by-proxy Jenny McCarthy, who almost single-handedly brought back such popular diseases like Measles and Mumps solely on the basis that her “mommy instincts” outweigh well-established scientific modalities, is one such person. I developed and occasionally teach a course on Information Management that uses the “Monty Hall Problem” (some call it a “paradox”, but I argue that there are no such things as true paradoxes. That is a subject of another post…) to demonstrate that SOME things which appear to be bone-headedly obvious might not be true at all - no matter how much they seem so. Friedrich Nietzsche summed this concept up beautifully when he said:
A casual stroll through the lunatic asylum shows us that faith proves nothing.
On hearing my enthusiasm for this quotation, one may invoke Voltaire, who said, “A witty saying proves nothing”, which does apply to something like, “The best things in life are free”, because one could easily challenge the meaning of “free” or provide examples of favorite things that are not free. However as witty as Nietzsche’s saying is, it isn’t so shallow as that. He states no premises that require substanciation. Regardless of your beliefs or your feelings towards Mr. Nietzsche, this statement stands on its own in that the logic is valid and we all understand the premises (that most if not all of the lunatics are delusional in spite of their personal convictions, a point logically proven if they disagree with each other) to be true – axiomal, if you will. If you question my statement about the premises, please reply with an explanation why.
I was first impressed with the validity of this sort of logic (and became aware of phenomenological arguments and post-modern thinking…both of which are despicable, but that may be the topic of a future post) when I first saw John Carpenter’s wonderful 1974 sci-fi film “Dark Star“, in which the crew of a spaceship attempts to convince an artificially-intelligent bomb not to explode (click to watch video of this scene); the problem being that the bomb-bay doors have failed to open but due to a communication glitch, the bomb is convinced that it has already been dropped and therefore MUST explode in order to “fulfill its destiny”. To make matters worse, the crew cannot manually stop the countdown…only the sentient bomb has the ability to do this. Unfortunately, no matter how hard the crew tries, they cannot convince the bomb to stop the countdown because they cannot convince the bomb that it hasn’t been dropped: all of it’s (damaged) sensors says that is is on it’s way to its target.
Acting Captain Doolittle decides that the best strategy is to “teach it phenomenology”, whereby they enlighten the bomb to the idea that our perception of the universe is a product not of actual reality, but the information that our senses provide to us…and that information could be wrong, therefore the bomb should question the order to explode. Although phenomenology usually doesn’t get you anywhere, Doolittle capitalized on that aspect when he used it to confuse the bomb into questioning its objective to explode:
DOOLITTLE: How do you know you exist?
BOMB #20: It is intuitively obvious.
DOOLITTLE: Intuition is no proof. What concrete evidence do you have that you exist?
BOMB #20: Hmmmm… well… I think, therefore I am…my sensory apparatus reveals it to me.
DOOLITTLE: How do you know that the evidence your sensory apparatus reveals to you is correct? What I’m getting at is this: the only experience that is directly available to you is your sensory data. And this data is merely a stream of electrical impulses which stimulate your computing center.
BOMB #20: In other words, all I really know about the outside universe relayed to me through my electrical connections. Why, that would mean…I really don’t know what the outside universe is like at all, for certain.
Once the bomb understands this idea, it asks the obvious question…
BOMB #20: True, but since this is so, I have no proof that you are really telling me all this.
…but Doolittle eloquently explains…
DOOLITTLE: That’s all beside the point. THE CONCEPTS ARE VALID, WHEREVER THEY ORIGINATE!
…and such is the power of internally-consistent logic.
Back to the Nietzsche quotation – his point was basically this: although faith may provide oneself with INTERNAL CERTAINTY (to know…), that does not necessarily translate to EXTERNAL ACCURACY (…but be wrong anyway) as demonstrated by the mutually exclusive, and often disproven, beliefs of the asylum tenants. Or, as Mark Knopfler of Dire Straits sang in “Industrial Disease”:
Two men say they’re Jesus…ONE of them must be wrong!
But I digress; For the purpose of this post, perhaps we should re-word Nietzsche’s statement as…
A casual look at the history of the Monty Hall Problem shows us that intuition proves nothing.
The challenge of opening people’s eyes to the idea that their cherished instincts can be wrong is that unless you actually gain THEIR acceptance of your position, you have failed. It is one thing to show someone up in a debate in front of an audience and convince the fence-sitters that your idea is supported by the evidence, but it is quite another thing to win the hearts and minds of those who are dug-in to their opposing view (ie – the dogmatic opponent). In fact, the advice usually given to scientists that are about to debate pseudo-scientists – those who abide by fallacious logic and manufactured evidence for a living – is that they shouldn’t even bother trying or they will end up being the one in the insane asylum. But for those up to the challenge, this post provides a means of confronting the power of intuition head-on.
What is “The Monty Hall Problem”?
For those who are unaware of the by-now-very-well-known “Monty Hall Problem” or have seen the movie “21”, I will briefly describe it here. Skip the next three paragraphs to tthe asterix* if you know the problem and solution, even if you don’t agree with the consensus opinion…
In the very real 1970s game show, “Let’s Make a Deal”, the host (actually fellow Canadian Monty Hall – no relation) presents to you three closed doors that are non-descript except for their being numbered 1, 2 & 3.
The question is this: Is it statistically advantageous to…
(a) stay with the original choice of door #1
(b) switch to unselected door #2 or
(c) do niether (a) nor (b), because it makes no difference?
The most common answer, based on the intuitive conclusion that if it’s either #1 or #2 then there must be an EQUAL chance of the big prize being behind either door, is “(c) – It makes no difference”.
Less common but in my experience still quite common is response “(a) stay with the original choice”, however this response typically comes from people who confuse “statistically advantageous” with “I commit to my decisions”.
Although wrong, the former (c) group tend to be smart people who, like most of us, are simply unaware of the actual statistics at play here (until recently even Nobel-prize-winning mathematicians got it wrong). However, the latter (a) group tend to be dogmatic-minded magical-thinking dumbf**ks that consider their random guesses to somehow transcend logic and evidence.
By elimination (and by statistical proof) the correct answer is “(b) switch to the unselected door”.
- The simplest explanation: When you start out, each door has a 1/3 chance of being a winner.
When you select a door, the chances yours is a winner is 1/3 and therefore the chances that the prize is behind a door you didn’t select is 2/3. Everyone knows that AT LEAST ONE of the two unselected doors is a loser, so when Mr. Hall opens it, he does not change the 1/3 chance that the door you selected is a winner. Rather, he has merely let you know which of the two unselected doors would have been a bad choice. Therefore, the chances that the prize is behind the other unselected door is 2/3 because 1/3+2/3=1, where the “1″ represents the 100% chance that the prize is SOMEWHERE behind the 3 doors.
- Google the problem and look at other (mathematical, logical, trials) solutions. If you don’t like them, then it sucks to be you – so get with it…this problem has swung from a sizable amount of of the mathematical community having previously being convinced that it doesn’t matter which door you pick back when the problem was first identified over 20 years ago, to presently the vast majority (there are still some fringe holdouts amongst mathematicians, like those very few ”scientists” that are also creationists – ref “Project Steve“, but these are handy a litmus-tests for incompetence as far as I’m concerned) of generally the SAME COMPETENT PEOPLE convinced that it is statistically advantageous (66% likelihood of winning) if you switch doors, after both doing the relevant math and through real-world test trials required by the last of the hold-outs.
- I, too ONCE swore that the answer was (a) and it wasn’t until I had an epiphany* that I came to not accept but understand, INTUITIVELY, that the correct answer is indeed (b). This was a paradigm-shifting moment for me because at that instant, any unfounded faith I may have had in anything (not that I had much to begin with, mind you) immediately evaporated and I became a “born-again” skeptic.
- It simply works: My newfound appreciation of the problem provided me with elevated powers to screw with people (I will get to this later).
*No, that wasn’t her stripper name. The following is the epiphany I mention in my statement #3 above…
- Imagine you are at your local convenience store and the proprietor has the only lot of 10 scratch-&-win tickets. Let’s say that he lost his licence to sell State-run lottery tickets and in order to stay in the business he is running his own (fair) lottery.
- One of the $10 tickets has a big-money payout ($90, which supposedly guarantees the proprietor a profit of $10, but of course only if he sells them all and continues to do so once someone wins). The remainder are worthless.
- As he was the creator of this special lottery, it is no secret that the proprietor already knows which of the tickets is good for the prize.
- You randomly select a ticket from the pile and pay him the $10 for it.
- The proprietor, who was stupid enough to lose his licence with the State and also stupid enough to think he will make a profit on the game if he (likely) sells the winning ticket before the 90th sale, offers to make the game “a little more interesting”.
- He says, “give me $60 and I will increase the chances of your winning the $100 from 10% to 50%. Although the statistics of this is NOT quite in your favour, you were dumb enough to buy a lottery ticket in the first place (or perhaps you are just REALLY curious and want to see where this is going), you accept the challenge and give him $60 for a supposed 50/50 chance to not quite double your money.
- Knowing that there are AT LEAST 8 losing tickets left in his pile (9, if you picked the winner) as well as which ticket is the actual winner, he scratches 8 of the unselected tickets, leaving one unscratched ticket in your hand and one in front of him on the table. Both of you know that one of these is the winner.
- He says to you, “Now, you can have either the one in your hand OR the one on the table. Which one do you want?” It is now clear that he only and erroneously THOUGHT it was 50/50 odds.
- However, you remember that when you started the game, the chances of the ticket you originally chose being the winner was only 10% and therefore deduce that the chances of the winning ticket remaining in the pile was 90%.
- When the proprietor scratched those 8 losing tickets, the original conditions never changed: The chances that the winning ticket remaining on the table continues to be 90%: he simply separated the wheat from the chaff and chances are that he has inadvertently told you where to MOST LIKELY find the winning ticket.
- You happily swap tickets knowing that there is a 90% chance that your $70 investment returns the $100 prize for a profit of $30.
- Now, imagine this same game with only three doors…Oops…tickets.
Go have a drink and a think and then come back to read on…
Throughout the entire run of “Let’s Make a Deal”, nobody understood the actual statistics behind the game. It would have only taken one wary statistician to bring this to light in order for the producers to feel the economic pressure to change the rules. It wasn’t until years after it was cancelled that someone figured it out, and she was the object of extreme and unwarranted (but thankfully, not lasting) professional revulsion for years after that.
This post has become absurdly long, so I am breaking it into three parts. Hopefully you now have greater appreciation for not only the elegance and practicality of the correct answer, but also – if the correct answer was a surprise to you – appreciate the implication this has on intuition-based belief sytems, which include faith.
In Part 2 of this series, I will tell the story about how I used the “Monty Hall Problem” to swindle an over-confident employer into sending me to the TAM7 conference in Las Vagas and in Part 3 I will describe my own version, which I call “The JASON Hall Problem” (I’m not an egomaniac, I just couldn’t resist the name), which is an improvement on the original in that it changes the most common intuitive answer from “stay or switch: it doesn’t matter” to “definitely stay”, whilst the correct answer is still to switch.
Comments are welcome and encouraged. However, before you post, please read my Moderation Policy, which I’ve adopted to control Spam. Basically, if you link to another website AND you do NOT refer to some specific detail about my post or another commenter’s post, your comment will be trashed before it appears, even if you are kind enough to say only, “I like your blog”. Sorry ’bout that, but the spam-bots have wrecked it for all of us.