Let \(X\) be a random variable which takes only the finite number of different possible real values \(x_{1},x_{2},\ldots,x_{n}.\) Define the expectation \(\mathbb{E}(X)\) and the variance \(\var(X)\) of \(X\). Show that , if \(a\) and \(b\) are real numbers, then \(\E(aX+b)=a\E(X)+b\) and express \(\var(aX+b)\) similarly in terms of \(\var(X)\). Let \(\lambda\) be a positive real number. By considering the contribution to \(\var(X)\) of those \(x_{i}\) for which \(\left|x_{i}-\E(X)\right|\geqslant\lambda,\) or otherwise, show that \[ \mathrm{P}\left(\left|X-\E(X)\right|\geqslant\lambda\right)\leqslant\frac{\var(X)}{\lambda^{2}}\,. \] Let \(k\) be a real number satisfying \(k\geqslant\lambda.\) If \(\left|x_{i}-\E(X)\right|\leqslant k\) for all \(i\), show that \[ \mathrm{P}\left(\left|X-\E(X)\right|\geqslant\lambda\right)\geqslant\frac{\var(X)-\lambda^{2}}{k^{2}-\lambda^{2}}\,. \]
Solution: Definition: \(\displaystyle \mathbb{E}(X) = \sum_{i=1}^n x_i \mathbb{P}(X = x_i)\) Definition: \(\displaystyle \mathrm{Var}(X) = \sum_{i=1}^n (x_i-\mathbb{E}(X))^2 \mathbb{P}(X = x_i)\) Claim: \(\mathbb{E}(aX+b) = a\mathbb{E}(X)+b\) Proof: \begin{align*} \mathbb{E}(aX+b) &= \sum_{i=1}^n (ax_i+b) \mathbb{P}(X = x_i) \\ &= a\sum_{i=1}^n x_i \mathbb{P}(X = x_i) + b\sum_{i=1}^n \mathbb{P}(X = x_i)\\ &= a \mathbb{E}(X) + b \end{align*} Claim: \(\mathrm{Var}(aX+b) = a^2 \mathrm{Var}(X)\) Claim: \(\mathrm{P}\left(\left|X-\mathrm{E}(X)\right|\geqslant\lambda\right)\leqslant\frac{\mathrm{var}(X)}{\lambda^{2}}\) Proof: \begin{align*} \mathrm{Var}(X) &= \sum_{i=1}^n (x_i-\mathbb{E}(X))^2 \mathbb{P}(X = x_i) \\ &\geq \sum_{|x_i - \mathbb{E}(X)| \geq \lambda} (x_i-\mathbb{E}(X))^2 \mathbb{P}(X = x_i) \\ &\geq \sum_{|x_i - \mathbb{E}(X)| \geq \lambda} \lambda^2 \mathbb{P}(X = x_i) \\ &= \lambda^2 \sum_{|x_i - \mathbb{E}(X)| \geq \lambda} \mathbb{P}(X = x_i) \\ &= \lambda^2 \mathrm{P}\left(\left|X-\mathrm{E}(X)\right|\geqslant\lambda\right) \end{align*} Claim: \[ \mathrm{P}\left(\left|X-\mathrm{E}(X)\right|\geqslant\lambda\right)\geqslant\frac{\mathrm{var}(X)-\lambda^{2}}{k^{2}-\lambda^{2}}\,. \] Proof: \begin{align*} && \mathrm{Var}(X) &= \sum_{i=1}^n (x_i-\mathbb{E}(X))^2 \mathbb{P}(X = x_i) \\ &&&= \sum_{|x_i - \mathbb{E}(X)| \geq \lambda} (x_i-\mathbb{E}(X))^2 \mathbb{P}(X = x_i) + \sum_{|x_i - \mathbb{E}(X)| < \lambda} (x_i-\mathbb{E}(X))^2 \mathbb{P}(X = x_i) \\ &&& \leq \sum_{|x_i - \mathbb{E}(X)| \geq \lambda} k^2 \mathbb{P}(X = x_i) + \sum_{|x_i - \mathbb{E}(X)| < \lambda} \lambda^2 \mathbb{P}(X = x_i) \\ &&&= k^2 \mathbb{P}\left(\left|X-\mathrm{E}(X)\right|\geqslant\lambda\right) + \lambda^2 \mathbb{P}\left(\left|X-\mathrm{E}(X)\right| < \lambda\right) \\ &&&= k^2 \mathbb{P}\left(\left|X-\mathrm{E}(X)\right|\geqslant\lambda\right) + \lambda^2(1- \mathbb{P}\left(\left|X-\mathrm{E}(X)\right| \leq \lambda\right) \\ &&&= (k^2 - \lambda^2) \mathbb{P}\left(\left|X-\mathrm{E}(X)\right|\geqslant\lambda\right) + \lambda^2 \\ \Rightarrow&& \frac{\mathrm{Var}(X)-\lambda^2}{k^2 - \lambda^2} &\leq \mathbb{P}\left(\left|X-\mathrm{E}(X)\right|\geqslant\lambda\right) \end{align*} [Note: This result is known as Chebyshev's inequality, and is an important starting point to understanding the behaviour of tails of random variables]
Whenever I go cycling I start with my bike in good working order. However if all is well at time \(t\), the probability that I get a puncture in the small interval \((t,t+\delta t)\) is \(\alpha\,\delta t.\) How many punctures can I expect to get on a journey during which my total cycling time is \(T\)? When I get a puncture I stop immediately to repair it and the probability that, if I am repairing it at time \(t\), the repair will be completed in time \((t,t+\delta t)\) is \(\beta\,\delta t.\) If \(p(t)\) is the probability that I am repairing a puncture at time \(t\), write down an equation relating \(p(t)\) to \(p(t+\delta t)\), and derive from this a differential equation relating \(p'(t)\) and \(p(t).\) Show that \[ p(t)=\frac{\alpha}{\alpha+\beta}(1-\mathrm{e}^{-(\alpha+\beta)t}) \] satisfies this differential equation with the appropriate initial condition. Find an expression, involving \(\alpha,\beta\) and \(T\), for the time expected to be spent mending punctures during a journey of total time \(T\). Hence, or otherwise, show that, the fraction of the journey expected to be spent mending punctures is given approximately by \[ \quad\frac{\alpha T}{2}\quad\ \mbox{ if }(\alpha+\beta)T\text{ is small, } \] and by \[ \frac{\alpha}{\alpha+\beta}\quad\mbox{ if }(\alpha+\beta)T\text{ is large.} \]