Probability Density Functions

Let $X$ be a random variable (r.v.). We say that $X$ is continuous if the probability of any fixed value $x$ is 0, i.e. $Pr(X=x)=0$, while the probability that $X$ takes one of the values in some interval $[a,b]$, or $(a,b)$, may be positive. We describe the probability law of such a variable in terms of its distribution function (d.f.) $F(x)=Pr(X\leq x)$, $-\infty < x < \infty$. As we know, $F$ is always right-continuous and monotone, increasing (in fact, non-decreasing) from value $0$ at $-\infty$ to value $1$ at $\infty .$

We assume more, that the d.f. $F$ obeys a density function, say $f$. This means that

Recall that $f$ is a non-negative function and the total integral of $f$ is equal to 1.

With $F$ or $f$ at hand, the probability that $X$ takes a value in a given interval $[a,b]$, $a<b$, is

The mean value (expectation) of a ‘nice’ function $g(X)$ of the r.v. $X$ is defined by

Transformations of Random Variables

Consider a random variable $X$ with p.d.f. $f_{X}(x)$. The p.d.f. of an arbitrary differentiable invertible transformation $Y=g(X)$ can be deduced as

Note that the term $\left|\frac{d}{dy}g^{-1}(y)\right|$ is known as the “Jacobian” of the transformation.

Normal Random Variables

• We say that the r.v. $X$ has a normal distribution with parameters $\mu$ and $\sigma ^2$, $X \ \sim \ {\textrm{N}}(\mu , \sigma ^2)$, if $X$ has the density function

  $$f(x)=\frac{1}{ \sqrt{2\pi }\sigma }\exp \left[-\frac{(x-\mu )^2}{2\sigma ^2}\right], \ -\infty < x < \infty ; \ \mu \ \mbox{ is any real}, \ \sigma > 0.$$

  This density is also called normal or Gaussian. The expectation and the variance of $X$ are

  $$E(X)=\mu ,\, \, \, \, {\rm Var}(X)= \sigma ^2.$$

  The graph of $f(x), \ -\infty < x< \infty$, is the familiar bell-shaped curve, symmetric about the axis $x= \mu$.

• A normal r.v. is called standard if $E(X)=0$ and ${\rm Var}(X)=1.$ In this case we use the notation $Z \ \sim {\textrm{N}}(0,1)$. That is to say, the density of $Z$ is

  $$\phi (x)=\frac{1}{\sqrt{2\pi }}e^{-x^2/2}, \ -\infty < x < \infty .$$

  The graph of $\phi$ is symmetric about the $y$-coordinate axis. The corresponding d.f. is

  $$\Phi (x)= \int _{-\infty }^x \frac{1}{\sqrt{2\pi }}e^{-y^2/2}dy, \ -\infty < x < \infty$$

  It is is called the standard normal (or Gaussian) distribution function.

Useful Properties of Normal Random Variables

• Linear Combinations: If the r.v. $X$ is normal, then so is $aX+b$, where $a,b$ are constants. If $X$ has mean $\mu$ and variance $\sigma ^2$, then $Z=(X-\mu )/\sigma$ is standard normal. (Can you check this?) This fact enables us to express probabilities related to $X$ in terms of $\Phi$.

• Independence of random variables: We say that the r.v.s $X_1,\ldots ,X_n$ are independent if for arbitrary intervals $I_1,\ldots ,I_n$, where $I_j=[a_j,b_j]$, closed or open, we have

  $$P[X_1\in I_1,\ldots ,X_n\in I_n]=P[X_1\in I_1]\cdot \ldots \cdot P[X_n\in I_n].$$

  If independent r.v.s $X_j$ are normal with parameters $\mu _j,\sigma _j^2,$ $j=1,\ldots ,n$, then the sum $X_1+\ldots +X_n$ is normal as well, with mean $\mu _1+\ldots +\mu _n$ and variance $\sigma ^2_1+\ldots +\sigma ^2_n$.

  In general, we have $E[X+Y]=E[X]+E[Y]$ for arbitrary r.v.s $X$ and $Y$. In contrast to this, the equality ${\rm Var}[X+Y]= {\rm Var}[X]+ {\rm Var}[Y]$ holds only if $X$ and $Y$ are uncorrelated (e.g. independent).