I wrote this thing on another blog and I thought it would be nice to write my findings here. For those who don't know it has to do with a twisted variation of the Monty Fall problem that bugged me for days. I first wrote about it in my previous blog entry. I am not the guy who spends a lot of time in endless discussions on twisted quizzes or counter-intuitive problems cause usually I am more practical, like who solves sudoku or IQ tests when I actually manage to write a new algorithm that is both clever and usefull? But this one for some reasons hit me and I became a little obsessed with it. I still don't know if it's 1/2 (I'd say, inspired by PHd comics that it's pi/2 :) but the simulation with my own rules does it even if the mathematicians prooved the problem differently (they didn't even defined if something happens when the host accidentally reveal the car,.. oh well just read at the end ps or just get familiarized with the monty hall/fall problems, except if you don't want to waste your time :)
For the Monty Fall problem (the second variation) there is too much controversy and that is I think because the description is kinda weird and I can't think of a way I could possibly simulate that in reality nor can I have a good insight (yet) of what is the difference or what hard math or a very deep perception of statistics or probability that I could possibly lack. Till now I thought this was also 2/3 and tried to find out why they say (Marilyn, the PDF with the probability math I still haven't read, etc) it's 1/2.
One think that is not explained is what happens if the host accidentally opened the car. This is not defined. It says that he slips in a banana and randomly happens to get the goat. So, does he always randomly slips and gets the goat (so that the game show makes sense) or is revealing the goat just a specific run (and what happens in this case) and in the other runs the host would possibly reveal the car too. Yet we are asked if in the specific run that a goat is revealed (but a car could be revealed in other runs too) what would happen?
Although to be able to check it in a simulation a rule must be set for what happens if the host chooses the car. So I invented my own two different rules for my simulation:
1) The unfair rule: The game goes on and the player switches from a goat to a goat and looses anyways. So, the host can accidentally reveal the car and make the player instantly loose. In a C programm the simulation really gets us the proposed 1/2. That's easy.
2) The rerun rule: If the host randomly reveals the car then that run is simply discarded (and not counted in the total runs) and we setup a new one and try again. Now in this one I initially made a 2/3 but when I explained the monty hall/fall problem to my brother he surprisingly came with nice ideas to use in my simulation. My mistake was that when I had a new run with the 3 doors, getting the car I started the simulation again but with the same contents in 3 doors. That means, if I had GOAT CAR GOAT and the host opened the 2nd door with the car, I reran with the same GOAT CAR GOAT (didn't randomized again a new set of doors) and randomly the host would select the 3rd door and let the run continue. But this was like canceling the mistake of the host and selecting always the goat, bringing it back to the old monty fall (not hall) problem, thus 2/3.
He actually told me:
CAR | GOAT GOAT
GOAT | CAR GOAT
GOAT | GOAT CAR
"In the occasion of the second or third row the host has a 50% probability to fuck up and select CAR. In the 1st row the mistake will never happen. So if you change your simulation programm so that a wrong choice actually reruns the game by shuffling the car/goats order in the doors, you will get twice times the first row than the others. In the first row if you switch you certainly loose. So you have two cases that switching makes you loose (two times the 1st row) and two other rows (2nd and 3rd) that switch makes you win. This is 2/4 aka 50%."
And so it actually worked in simulation for both rules getting 50%.
p.s. Although I don't think that the people who originally invented the Monty Fall version of the problem thought of these rules for the solution. If they had they would be more precise. Somehow I feel that the Monty Fall is a slight variant of the Monty Hall deliberately made to get a 1/2 effect maybe to justify the several PHDs who did it wrong on the first problem and say that, they understood the Monty Fall variation from the description, not the Monty Hall description. They say that Marilyn had an unclear description of the initial problem. But Marilyn's description is just plain right. It's the Monty Fall variation that has an unclear or misleading or not complete (needs more data) description for me. It's just a counter-intuitive problem that most of us reply 1/2 at first sight. Did all of us thought of the Monty Fall problem? No. We just didn't see the whole image, just hide the revealed car and worked independently of the initial choice. A common pitfall. I don't believe that all those PHDs and especially other non-mathematical people just got it different. They got it wrong with a first guess as I initially and my brother and my friends did.
Thursday, July 09, 2009
Tuesday, July 07, 2009
Mind twisting
It's been three days since I started obsessively analyzing a simple probability problem and different versions of it. Several years ago a friend told me the story of a woman who baffled several mathematicians by insisting in her extreme sounding solution which proved at the end to be true, and her case made an impression on me yet my friend couldn't remember her name or I didn't googled it enough then. And just few days ago I was ecstatic to find the story at Coding Horror.
I won't explain the problem since it's very nicely written in the link above and many people might be familiar already with it (please visit the link first if you aren't). It's just funny that after you read the problem you initially say 1/2 and it sounds so logical, you read 2/3 and you are certain that she is wrong, even though her explanations are quite simple and they are truly making sense (even to the math illiterate), while even famous mathematicians say this can't be right, till a call for trying to simulate a probability experiment proves them wrong. I tried the same thing in a C program and it really shows after few runs. There are even java applets that let you play the game of the problem for several runs and report the probabilities. Wow! (It's funny to read the story on the site of Marilyn and grin at the reactions regardless the simple mathematical or empirical explanations.
Of course I didn't described the problem but sent you into external links because it's not the one I want to discuss (it's already resolved) but the extension of it.
Say that you have the three doors again, you start with your choice and later the host opens one of the two remaining doors but randomly this time (not deliberately revealing the one with the goat). This means that he could also mistakenly open the door with the winning car. Although because in that case the show would be disaster it assumes that he gets lucky and randomly chooses a door with a goat. What's the possibility of switching from your initial choice to the alternative remaining one?
I would assume at first that since the host selected the goat, whether he did it deliberately or not this occurrence returns us back to the first version of the problem. You have taken one door and the host reveals a goat, your probability of switching is still 2/3. But both Marilyn, the wikipedia article and some mathematical pdf explanation states it's 1/2. And that's where the baffle begins.
First of all the example is quite imaginary and I could not easily think of a practical way to do the many runs and find out the experimental results. In the classic problem the host would deliberately select the door with the goat. So if he knew that one of the two doors contained the car he wouldn't chose it at all to not ruin the show. In the description of the extended problem the host forgets which door has the car and randomly chooses one of the two remaining doors, although it assumes that he luckily avoids choosing the door with the car. One could mistake that by thinking that even in several runs he always gets uberlucky to always not hit a car. But if I understood the description well you could assume that in that particular run he gets lucky and selects the goat yet the rules of the game is that he could even have chosen the car (someone would say that we don't care what would happen then since we only analyze this run). Yet I still have some good arguments why switching might not be 1/2.
First of all somebody should give an insight of what would happen if in a specific run the car is revealed. Because it matters if some of us wish to run a hundred of runs in experiment or computer simulation to be convinced. In that case, would the host say "ooops", pause the show to create a new arrangement of goat/car placements and start from scratch? This is like discarding the cases where he accidentally chooses the car so he always chooses the goat bringing us back to the initial problem with the 2/3 solution.
The second alternative would be that there is a special rule that says, if the host randomly reveals the car the competition goes on normally and then the player looses anyways. In my simulation program I removed the code that denies the host from revealing a door if it's a car. And then I don't care what would happen in real life if such a thing occurred and just run the simulation. A switch between the unrevealed doors will move from a goat to a goat and the player will loose anyways. Say that it's unfair rules of the show. This simulation gives a 1/2 after several runs. The problem though is that it doesn't suppose that a host slips in a banana and randomly reveals a door that happens to be a goat, etc, etc. It takes as valid that he reveals a car too.
There is something seriously wrong with the description of the second problem. It assumes that the host randomly chooses a door yet again it claims that it has to be a goat, yet it's still could be a car but it never is, while it doesn't claim what it would happen in the case it ways which isn't necessary seems we assume that it randomly is always (or in one run) a goat. I mean,.. it's as crazy as Schroedinger's cat!
A way it would make sense is to split it into four categories. Two of them happens before the game starts, the other two take place just after the host reveals a door.
I say to a friend that I want to go to a game show where at the end there are the three doors and the host always randomly reveals one after my initial choice. He speculates:
Case 1: If accidentally revealing a car forces the host to cancel this run and do it again from the beginning (and the next one accidental car choice, recursively forces him to discard the next run again) then the only valid run that finally happens is the one where he reveals a goat. This goes back to the original problem with the solution of 2/3.
Case 2: Accidentally revealing the car by the host results in the unfair rule of the player loosing. Either he switches or stays he gets the goat. Remember, I am not in the middle of the game, my friend speculates what are the possibilities either I switch or not based on the unfair rule. I don't know yet if he will select a goat or a car in the future. It tells me that if I play such a game in the future and given the possibility to switch, I have a probability of 1/2 to win either way.
I am already in the last part of the show, I have already chosen a door that I don't open yet and the host is about to reveal another one of the two. In the situation that the rule of case 1 was valid (discarding the revealing of the car mistake and doing it again) it would still fit in the old problem with the 2/3 solution. We only discuss now the situation when the unfair rule is at work.
Case 3: The host accidentally reveals the car. Either switch or stay gets a probability of zero.
Case 4: And now for the most important case. This is the one that is described in the problem in my opinion. The answer for this matters the most. The host randomly revealed the goat. He actually gave you an advantage! There was a possibility that he would hit case 3 but he didn't and your turn comes after that fact. He eliminated some negative odds of choosing the car concerning the unfair rules are at play. I can't think but the fact that it brings us to the old goat problem with 2/3 probability. While scientists, wikipedia and Marilyn says 1/2. This is where I am still baffled what am I thinking wrong!
It's hard to think that all those people have made a mistake again, so maybe I should have a look at this article (at the monty fall problem) and decide. I hope the theoritical math of this one can also give me a practical view of how this solution could apply and be explained using your perception in the real world. I'd like to see the theoritical proof and then see if somehow it also makes sense in reality. And how could someone create a probability experiment on this one? How to make the host randomly open the door yet he always chooses the goat? Doesn't this eliminate the other case of taking the car? Doesn't this converge our simulation to have several runs that look like the ones in the old problem bringing us the 2/3 result again?
If I am really wrong on this one then I would like to hear some proper explanations of why the 1/2 persists? The old problem sounded baffling but wasn't at all when you thought of the explanation. But the new one, if I get a proper answer it will either be something that changes my perception to something ever more crazy or the not so interesting yet revealing answer that the description of this problem and the way it's solution is suggested suffers from bad logic.
It will surely occupy my brain for more days. What a mind twister!
I won't explain the problem since it's very nicely written in the link above and many people might be familiar already with it (please visit the link first if you aren't). It's just funny that after you read the problem you initially say 1/2 and it sounds so logical, you read 2/3 and you are certain that she is wrong, even though her explanations are quite simple and they are truly making sense (even to the math illiterate), while even famous mathematicians say this can't be right, till a call for trying to simulate a probability experiment proves them wrong. I tried the same thing in a C program and it really shows after few runs. There are even java applets that let you play the game of the problem for several runs and report the probabilities. Wow! (It's funny to read the story on the site of Marilyn and grin at the reactions regardless the simple mathematical or empirical explanations.
Of course I didn't described the problem but sent you into external links because it's not the one I want to discuss (it's already resolved) but the extension of it.
Say that you have the three doors again, you start with your choice and later the host opens one of the two remaining doors but randomly this time (not deliberately revealing the one with the goat). This means that he could also mistakenly open the door with the winning car. Although because in that case the show would be disaster it assumes that he gets lucky and randomly chooses a door with a goat. What's the possibility of switching from your initial choice to the alternative remaining one?
I would assume at first that since the host selected the goat, whether he did it deliberately or not this occurrence returns us back to the first version of the problem. You have taken one door and the host reveals a goat, your probability of switching is still 2/3. But both Marilyn, the wikipedia article and some mathematical pdf explanation states it's 1/2. And that's where the baffle begins.
First of all the example is quite imaginary and I could not easily think of a practical way to do the many runs and find out the experimental results. In the classic problem the host would deliberately select the door with the goat. So if he knew that one of the two doors contained the car he wouldn't chose it at all to not ruin the show. In the description of the extended problem the host forgets which door has the car and randomly chooses one of the two remaining doors, although it assumes that he luckily avoids choosing the door with the car. One could mistake that by thinking that even in several runs he always gets uberlucky to always not hit a car. But if I understood the description well you could assume that in that particular run he gets lucky and selects the goat yet the rules of the game is that he could even have chosen the car (someone would say that we don't care what would happen then since we only analyze this run). Yet I still have some good arguments why switching might not be 1/2.
First of all somebody should give an insight of what would happen if in a specific run the car is revealed. Because it matters if some of us wish to run a hundred of runs in experiment or computer simulation to be convinced. In that case, would the host say "ooops", pause the show to create a new arrangement of goat/car placements and start from scratch? This is like discarding the cases where he accidentally chooses the car so he always chooses the goat bringing us back to the initial problem with the 2/3 solution.
The second alternative would be that there is a special rule that says, if the host randomly reveals the car the competition goes on normally and then the player looses anyways. In my simulation program I removed the code that denies the host from revealing a door if it's a car. And then I don't care what would happen in real life if such a thing occurred and just run the simulation. A switch between the unrevealed doors will move from a goat to a goat and the player will loose anyways. Say that it's unfair rules of the show. This simulation gives a 1/2 after several runs. The problem though is that it doesn't suppose that a host slips in a banana and randomly reveals a door that happens to be a goat, etc, etc. It takes as valid that he reveals a car too.
There is something seriously wrong with the description of the second problem. It assumes that the host randomly chooses a door yet again it claims that it has to be a goat, yet it's still could be a car but it never is, while it doesn't claim what it would happen in the case it ways which isn't necessary seems we assume that it randomly is always (or in one run) a goat. I mean,.. it's as crazy as Schroedinger's cat!
A way it would make sense is to split it into four categories. Two of them happens before the game starts, the other two take place just after the host reveals a door.
I say to a friend that I want to go to a game show where at the end there are the three doors and the host always randomly reveals one after my initial choice. He speculates:
Case 1: If accidentally revealing a car forces the host to cancel this run and do it again from the beginning (and the next one accidental car choice, recursively forces him to discard the next run again) then the only valid run that finally happens is the one where he reveals a goat. This goes back to the original problem with the solution of 2/3.
Case 2: Accidentally revealing the car by the host results in the unfair rule of the player loosing. Either he switches or stays he gets the goat. Remember, I am not in the middle of the game, my friend speculates what are the possibilities either I switch or not based on the unfair rule. I don't know yet if he will select a goat or a car in the future. It tells me that if I play such a game in the future and given the possibility to switch, I have a probability of 1/2 to win either way.
I am already in the last part of the show, I have already chosen a door that I don't open yet and the host is about to reveal another one of the two. In the situation that the rule of case 1 was valid (discarding the revealing of the car mistake and doing it again) it would still fit in the old problem with the 2/3 solution. We only discuss now the situation when the unfair rule is at work.
Case 3: The host accidentally reveals the car. Either switch or stay gets a probability of zero.
Case 4: And now for the most important case. This is the one that is described in the problem in my opinion. The answer for this matters the most. The host randomly revealed the goat. He actually gave you an advantage! There was a possibility that he would hit case 3 but he didn't and your turn comes after that fact. He eliminated some negative odds of choosing the car concerning the unfair rules are at play. I can't think but the fact that it brings us to the old goat problem with 2/3 probability. While scientists, wikipedia and Marilyn says 1/2. This is where I am still baffled what am I thinking wrong!
It's hard to think that all those people have made a mistake again, so maybe I should have a look at this article (at the monty fall problem) and decide. I hope the theoritical math of this one can also give me a practical view of how this solution could apply and be explained using your perception in the real world. I'd like to see the theoritical proof and then see if somehow it also makes sense in reality. And how could someone create a probability experiment on this one? How to make the host randomly open the door yet he always chooses the goat? Doesn't this eliminate the other case of taking the car? Doesn't this converge our simulation to have several runs that look like the ones in the old problem bringing us the 2/3 result again?
If I am really wrong on this one then I would like to hear some proper explanations of why the 1/2 persists? The old problem sounded baffling but wasn't at all when you thought of the explanation. But the new one, if I get a proper answer it will either be something that changes my perception to something ever more crazy or the not so interesting yet revealing answer that the description of this problem and the way it's solution is suggested suffers from bad logic.
It will surely occupy my brain for more days. What a mind twister!
Subscribe to:
Posts (Atom)
