Write down the probability of obtaining \(k\) heads in \(n\) tosses of a fair coin. Now suppose that \(k\) is known but \(n\) is unknown. A maximum likelihood estimator (MLE) of \(n\) is defined to be a value (which must be an integer) of \(n\) which maximizes the probability of \(k\) heads. A friend has thrown a fair coin a number of times. She tells you that she has observed one head. Show that in this case there are two MLEs of the number of tosses she has made. She now tells you that in a repeat of the exercise she has observed \(k\) heads. Find the two MLEs of the number of tosses she has made. She next uses a coin biased with probability \(p\) (known) of showing a head, and again tells you that she has observed \(k\) heads. Find the MLEs of the number of tosses made. What is the condition for the MLE to be unique?
Solution: \begin{align*} && \mathbb{P}(k \text{ heads} | n\text{ tosses}) &= \binom{n}k 2^{-n} \\ && \mathbb{P}(1 \text{ head} | n\text{ tosses}) &= n2^{-n} \\ \Rightarrow && \frac{ \mathbb{P}(1 \text{ head} | n+1\text{ tosses}) }{ \mathbb{P}(1 \text{ head} | n\text{ tosses}) } &= \frac{n+1}{2n} \end{align*} Which is less than \(1\) unless \(n \geq 1\). Therefore the MLE is \(n = 1\) or \(n= 2\). \begin{align*} \frac{ \mathbb{P}(k \text{ head} | n+1\text{ tosses}) }{ \mathbb{P}(k \text{ head} | n\text{ tosses}) } &= \frac{\binom{n+1}{k}}{2 \binom{n}{k}} \\ &= \frac{(n+1)!(n-k)!}{2n!(n+1-k)!} \\ &= \frac{n+1}{2(n+1-k)} \end{align*} This is less than or equal to \(1\) if \(n+1 = 2(n+1-k) \Leftrightarrow n= 2k-1\), therefore the MLEs are \(2k-1\) and \(2k\). If the coin is biased, we have \begin{align*} && \frac{ \mathbb{P}(k \text{ head} | n+1\text{ tosses}) }{ \mathbb{P}(k \text{ head} | n\text{ tosses}) } &= \frac{\binom{n+1}{k}p^kq^{n+1-k}}{\binom{n}{k}p^kq^{n-k}} \\ &&&= \frac{n+1}{(n+1-k)}q \\ \\ && 1 & \geq \frac{n+1}{(n+1-k)}q \\ \Leftrightarrow && (n+1)(1-q) &\geq k \\ \Leftrightarrow && n+1 & \geq \frac{k}{p} \end{align*} Therefore the probability is increasing until \(n+1 \geq \frac{k}{p}\). If \(\frac{k}p\) is an integer the MLEs are \(\frac{k}{p}-1\) and \(\frac{k}p\), otherwise it is \(\lfloor \frac{k}{p} \rfloor\) and the MLE is unique.
A hostile naval power possesses a large, unknown number \(N\) of submarines. Interception of radio signals yields a small number \(n\) of their identification numbers \(X_i\) (\(i=1,2,...,n\)), which are taken to be independent and uniformly distributed over the continuous range from \(0\) to \(N\). Show that \(Z_1\) and \(Z_2\), defined by $$ Z_1 = {n+1\over n} {\max}\{X_1,X_2,...,X_n\} \hspace{0.3in} {\rm and} \hspace{0.3in} Z_2 = {2\over n} \sum_{i=1}^n X_i \;, $$ both have means equal to \(N\). Calculate the variance of \(Z_1\) and of \(Z_2\). Which estimator do you prefer, and why?