1 problem found
You play the following game. You throw a six-sided fair die repeatedly. You may choose to stop after any throw, except that you must stop if you throw a 1. Your score is the number obtained on your last throw. Determine the strategy that you should adopt in order to maximize your expected score, explaining your reasoning carefully.
Solution: Once you have thrown, all previous throws are irrelevant so the only thing which can affect your decision is the current throw. Therefore the strategy must consist of a list of states we re-throw from, and a list of states we stick on. It must also be the case that if we stick on \(k\) we stick on \(k+1\) (otherwise we can improve our strategy by switching those two values around). Therefore we can form a table of our expected score: \begin{array}{c|c|c} \text{stop on} & \text{possible outcomes} & \E[\text{score}] \\ \hline \geq 2 & \{1,2,3,4,5,6\} & \frac{21}{6} = 3.5 \\ \geq 3 & \{1,3,4,5,6\} & \frac{19}{5} = 3.8 \\ \geq 4 & \{1,4,5,6\} & \frac{16}{4} = 4 \\ \geq 5 & \{1,5,6\} & \frac{12}{3} = 4 \\ =6 & \{1,6\} & \frac{7}{2} = 3.5 \end{array} Therefore the optimal strategy is to stop on \(4\) or higher. If we cared about variance we might look at the variance of the two best strategies, \(4\) or higher has a variance of \(\frac{1+16+25+36}{4} - 16 = 3.5\) and \(5\) or higher has a variance of \(\frac{1+25+36}3 - 16 = \frac{14}3 > 3.5\) so \(4\) or higher is probably better in most scenarios.