Problem source

\begin{questionparts}
\item I toss a biased coin which has a probability
$p$ of landing heads and a probability $q=1-p$ of landing tails.
Let $K$ be the number of tosses required to obtain the first head
and let 
\[
\mathrm{G}(s)=\sum_{k=1}^{\infty}\mathrm{P}(K=k)s^{k}.
\]
Show that 
\[
\mathrm{G}(s)=\frac{ps}{1-qs}
\]
and hence find the expectation and variance of $K$. 
\item I sample cards at random with replacement from a normal
pack of $52$. Let $N$ be the total number of draws I make in order
to sample every card at least once. By expressing $N$ as a sum $N=N_{1}+N_{2}+\cdots+N_{52}$
of random variables, or otherwise, find the expectation of $N$. Estimate
the numerical value of this expectation, using the approximations
$\mathrm{e}\approx2.7$ and $1+\frac{1}{2}+\frac{1}{3}+\cdots+\frac{1}{n}\approx0.5+\ln n$
if $n$ is large.
\end{questionparts}

Solution source

\begin{questionparts}
\begin{align*}
&& G(s) &= \sum_{k=1}^{\infty}\mathbb{P}(K=k)s^{k} \\
&&&= \sum_{k=1}^{\infty}q^{k-1}p\cdot s^{k} \\
&&&= \frac{p}q\sum_{k=1}^{\infty}(qs)^{k} \\
&&&= \frac{p}{q} \cdot \frac{qs}{1-qs} \\
&&&= \frac{ps}{1-qs}
\end{align*}

Not that $G$ is the pgf for $K$ and we can find $\E[K] = G'(1)$ so

\begin{align*}
&& G'(s) &= \frac{p(1-qs)+qps}{(1-qs)^2} \\
&&&= \frac{p}{(1-qs)^2} \\
\Rightarrow && G'(1) &= \frac{p}{p^2} = \frac1p \\
\end{align*}

Also notice that $\E[K(K-1)] = \E[K^2]-\E[K] = G''(1)$ so

\begin{align*}
&& G''(s) &= \frac{2pq}{(1-qs)^3} \\
\Rightarrow && G''(1) &= \frac{2pq}{p^3} = \frac{2q}{p^2} \\
\Rightarrow && \var[K] &= G''(1) + G'(1) - (G'(1))^2 \\
&&&= \frac{2q}{p^2} + \frac1p - \frac1{p^2} \\
&&&= \frac{2q+p-1}{p^2} \\
&&&= \frac{q}{p^2} = \frac{1-p}{p^2}
\end{align*}

\item Let $N_i$ be the number of draws between the $(i-1)$th new card and the $i$th new card. (Where $N_1 = 1$0 then $N_i \sim K$ with $p = \frac{53-i}{52}$). Therefore

\begin{align*}
\E[N] &= \E[N_1 + \cdots + N_{52}] \\
&= \E[N_1] + \cdots + \E[N_i] + \cdots + \E[N_{52}] \\
&= 1 + \frac{52}{51} + \cdots + \frac{52}{53-k} + \cdots + \frac{52}{1} \\
&= 52 \left (1 + \frac{1}{2} + \cdots + \frac{1}{52} \right) \\
&= 52 \cdot \left ( 1 + \ln 52 \right)
\end{align*} 

Notice that $2.7 \times 2.7 = 7.29$ and $7.3 \times 7.3 \approx 53.3$ so

$\ln 52 \approx 4$ and so our number is $\approx 52 \cdot 4.5 =234$.

[The correct answer actual number is 235.9782]
\end{questionparts}