Problem source

A group of biologists attempts to estimate the magnitude, $N$, 
of an island population of voles ({\it Microtus agrestis}).   
Accordingly, the biologists  capture a random sample of 200 voles, mark them
and release them. A second random sample of 200 voles is then taken
of which 11 are found to be marked.  Show that the probability, $p_N$, of
this occurrence is given by
$$
p_N = k{{{\big((N-200)!\big)}^2}
\over
{N!(N-389)!}},
$$
where $k$ is independent of $N$.
The biologists  then estimate $N$ by calculating the value of
$N$ for which $p_N$ is a maximum. Find this estimate.
All unmarked voles in the second sample  
are marked and then the entire sample is released.
Subsequently a third random sample of 200
voles is taken. Write down the probability
that this sample contains exactly $j$ marked voles, leaving your 
answer in terms of binomial coefficients.
Deduce that 
$$
      \sum_{j=0}^{200}{389 \choose j}{3247 \choose {200-j}}
= {3636 \choose 200}.
$$

Solution source

There will be $200$ marked vols out of $N$, and we are finding $11$ of them. There are $\binom{200}{11}$ ways to chose the $11$ marked voles and $\binom{N - 200}{200-11}$ ways to choose the unmarked voles. The total number of ways to choose $200$ voles is $\binom{N}{200}$. Therefore the probability is

    \begin{align*}
        p_N &= \frac{\binom{200}{11} \cdot \binom{N - 200}{200-11}}{\binom{N}{200}} \\
        &= \binom{200}{11} \cdot \frac{ \frac{(N-200)!}{(189)!(N - 389)!} }{\frac{N!}{(N-200)!(200)!}} \\
        &= \binom{200}{11} \frac{200!}{189!} \frac{\big((N-200)!\big)^2}{N!(N-389)!}
    \end{align*}

    As required and $k = \binom{200}{11} \frac{200!}{189!}$.

    We want to maximise $\frac{(N-200)!^2}{N!(N-389)!}$, we will do this by comparing consecutive $p_N$.

    \begin{align*}
        \frac{p_{N+1}}{p_N} &= \frac{\frac{(N+1-200)!^2}{(N+1)!(N+1-389)!}}{\frac{(N-200)!^2}{N!(N-389)!}} \\
        &= \frac{(N-199)!^2 \cdot N! \cdot (N-389)!}{(N+1)!(N-388)!(N-200)!^2} \\
        &= \frac{(N-199)^2 \cdot 1 \cdot 1}{(N+1) \cdot (N-388)\cdot 1} \\
    \end{align*}
    \begin{align*}
        && \frac{p_{N+1}}{p_N} &> 1 \\
        \Leftrightarrow && \frac{(N-199)^2 \cdot 1 \cdot 1}{(N+1) \cdot (N-388)\cdot 1} & > 1 \\
        \Leftrightarrow && (N-199)^2 & > (N+1) \cdot (N-388) \\
        \Leftrightarrow && N^2-2\cdot199N+199^2 & > N^2 - 387N -388 \\
        \Leftrightarrow && -398N+199^2 & > - 387N -388 \\
        \Leftrightarrow && 199^2+388 & > 11N\\
        \Leftrightarrow && \frac{199^2+388}{11} & > N\\
        \Leftrightarrow && 3635\frac{4}{11} & > N\\
    \end{align*}

    Therefore $p_N$ is increasing if $N \leq 3635$, so we should take $N = 3636$.

    \[ \P(\text{exactly } j \text{ marked voles}) = \frac{\binom{389}{j} \cdot \binom{3636 - 389}{200-j}}{\binom{3636}{200}}\]

    Since 
    
    \begin{align*}
        && 1 &= \sum_{j=0}^{200} \P(\text{exactly } j \text{ marked voles}) \\
        && &= \sum_{j=0}^{200}   \frac{\binom{389}{j} \cdot \binom{3247}{200-j}}{\binom{3636}{200}} \\ 
        \Leftrightarrow&& \binom{3636}{200} &= \sum_{j=0}^{200} \binom{389}{j} \cdot \binom{3247}{200-j} 
    \end{align*}