When reviewing the sample spaces related to probability theory, we saw that they can be of two types: discrete and continuous. We also reviewed what constitutes a discrete probability distribution. Now it is time to discuss continuous probability distributions.

What are continuous probability distributions?

We say that a random variable X has a continuous probability distribution if there exists a function f_X : \mathbb{R} \longrightarrow \mathbb{R}^+, which we will call the Density of X, such that \forall A \subseteq \mathbb{R} the following equality holds:

P(X\in A) = \displaystyle \int_A f_X(x)dx

In particular, if we take A=]a,b] we will have:

P(a\lt X \leq b) = \displaystyle \int_a^b f_X(x)dx

and if a=-\infty then:

F_X(x) = P( X \leq x) = \displaystyle \int_{-\infty}^x f_X(t)dt

Moreover, from property (c) of probability distributions we have that:

\displaystyle \int_{-\infty}^{+\infty} f_X(t)dt = 1

By applying the fundamental theorem of calculus to this last expression, we have that for a continuous distribution, F_X(x), is continuous for all x, and its derivative is f_X(x) for all x where f_X(x) is continuous. From the continuity of F_X(x) and property (d) (see here) it follows that:

P(x=X)=0

And therefore:

P(x\leq X)= P(x\lt X)

If f is any function that satisfies f\geq 0 and \displaystyle \int_{-\infty}^{+\infty}f(x)dx = 1, then it is called a density.

The 5 most well-known continuous probability distributions

Exponential Distribution

An exponential distribution function with parameter \alpha \gt 0 is a distribution function F of the form:

F(t) = \left\{\begin{array}{lll} 1 - e^{-t/\alpha} & ; & t\geq 0 \\ \\ 0 & ; & t\lt 0 \end{array}\right.

Consequently, its density function is of the form:

\displaystyle f(t) = \left\{\begin{array}{lll} \frac{1}{\alpha}e^{-t/\alpha} & ; & t\geq 0 \\ \\ 0 & ; & t\lt 0 \end{array}\right.

If a random variable has an exponential distribution with parameter \alpha we write X\sim Ex(\alpha).

In the context of the Poisson distribution, if we have a radioactive sample that emits a particle at an average emission rate c, then the time T at which the first particle is emitted has an exponential distribution with parameter 1/c. In other words, T\sim Ex(1/c), and consequently:

P(T\geq t)= e^{-ct}

Rectangular Uniform Distribution

A rectangular uniform distribution over an interval [a,b] is one that is defined by the density function:

f(x) = \left\{\begin{array}{lll} \displaystyle\frac{1}{b-a} & ; & x\in[a,b] \\ \\ 0 & ; & E.O.C. \end{array}\right.

If we drop a small ball on a rail with boundaries at the ends of the interval [a,b], and it bounces elastically off the edges, then the random variable X associated with the stopping position of the ball due to friction has a rectangular uniform distribution and is written X\sim Un(a,b).

Normal (Gaussian) Distribution

Among continuous distributions of probability, the normal distribution is one of the most popular in practice.

Standard Normal Distribution

The standard normal density is defined by the function:

\displaystyle \phi_{0,1}(x) = \frac{1}{\sqrt{2\pi}} e^{-x^2/2}

By its definition, it is clear that \phi\gt 0. Therefore, it can be verified that this is a probability density simply by confirming that:

\displaystyle \int_{-\infty}^{+\infty}\phi_{0,1}(x)dx

This last equality can be proven by calculating the value of I^2 when I =\int_{-\infty}^{+\infty}\phi(x)dx=1. In fact, it is:

\begin{array}{rl} I^2 & = \displaystyle \int_{-\infty}^{+\infty}\frac{1}{\sqrt{2\pi}} e^{-x^2/2} dx \int_{-\infty}^{+\infty}\frac{1}{\sqrt{2\pi}} e^{-x^2/2}dx \\ \\ & = \displaystyle \int_{-\infty}^{+\infty}\frac{1}{\sqrt{2\pi}} e^{-x^2/2} dx \int_{-\infty}^{+\infty}\frac{1}{\sqrt{2\pi}} e^{-y^2/2} dy \\ \\ & = \displaystyle \frac{1}{{2\pi}} \int_{-\infty}^{+\infty} \int_{-\infty}^{+\infty} e^{-\frac{x^2 + y^2}{2}} dxdy \\ \\ \end{array}

But it turns out that:

\displaystyle \int_{-\infty}^{+\infty} \int_{-\infty}^{+\infty} e^{-\frac{x^2 + y^2}{2}} dxdy = \int_{0}^{2\pi} \int_{0}^{+\infty} e^{-r^2/2} rdr d\theta = 2\pi

Therefore I^2 = 1, so that I=\int_{-\infty}^{+\infty}\phi_{0,1}(x)dx = 1.

From the standard normal density, the standard normal distribution is defined \Phi_{0,1}(x) = \int_{-\infty}^x\phi_{0,1}(t)dt. If a random variable X has a standard normal distribution, it is written X\sim N(0,1). The distribution \Phi_{0,1}(x) cannot be calculated explicitly, however, there are tables that allow quick approximate values to be obtained.

Normal distribution with parameters \mu and \sigma

From the standard normal distribution density \phi_{0,1} it is possible to construct the density for the normal distribution with parameters \mu and \sigma, where \mu\in\mathbb{R} and \sigma\gt 0 are, respectively, the mean and the standard deviation. The density of the normal distribution with these parameters is written as follows:

\displaystyle\phi_{\mu,\sigma}(x) = \frac{1}{\sigma}\phi_{0,1}\left(\frac{x-\mu}{\sigma} \right)

So the normal distribution with parameters \mu and \sigma, \Phi_{\mu,\sigma}(x), is of the form:

\displaystyle \Phi_{\mu,\sigma}(x) = \int_{-\infty}^x\frac{1}{\sigma}\phi_{0,1}\left(\frac{t-\mu}{\sigma} \right)dt = \frac{1}{\sqrt{2\pi\sigma}}\int_{-\infty}^x e^{-\frac{(t-\mu)^2}{2\sigma^2}}dt

If the random variable X has a normal distribution with parameters \mu, \sigma, then it is written X\sim N(\mu, \sigma).

Weibull Distribution

The Weibull distribution with parameters \alpha,\beta \gt 0 has a distribution function of the form:

F(t) = \left\{\begin{array}{llr} \left(1 - e^{-t/\alpha} \right)^\beta &;& t\geq 0 \\ \\ 0 &;& t\lt 0 \end{array}\right.

If a random variable X has a Weibull distribution with parameters \alpha, \beta it is written X\sim We(\alpha,\beta). The Weibull distribution is a generalization for the exponential distribution, note that We(\alpha,1) = Ex(\alpha).

Gamma Distribution

The Gamma distribution with parameters \beta,\alpha has a density function of the form:

f(t) = \left\{\begin{array}{llr} \displaystyle \frac{1}{\alpha \Gamma(\beta)}\left(\frac{t}{\alpha} \right)^{\beta-1}e^{-t/\alpha} &;& t\geq 0 \\ \\ 0 &;& t\lt 0 \end{array}\right.

Where \Gamma(s) = \displaystyle \int_0^{+\infty}u^{s-1}e^{-u}du is what is known as the “Gamma Function”.

One of the most notable properties of the Gamma function is that it allows the factorials of natural numbers to be generalized over the reals (and even the complex numbers). It is not difficult to verify that \Gamma(s+1) = s\Gamma(s) by integrating by parts. Moreover, since \Gamma(1)=1 it follows that:

\left(\forall n\in\mathbb{N}\right)\left(\Gamma(n) = (n-1)! \right)

If a random variable X has a Gamma distribution with parameters \beta, \alpha it is written X\sim Ga(\alpha,\beta). The Gamma distribution is another generalization for the exponential distribution, note that Ga(\alpha,1) = Ex(\alpha).

In a Poisson process with frequency c (such as radioactive decay), if T is the random variable representing the time when the m-th event occurs; then, given a t\geq 0 and a number N of events that occur in the time interval [0,t] we have that t\lt T \leftrightarrow N\lt m and, since N\sim Po(ct), we have:

1-F_T(t) = P(T\gt t) = \displaystyle \sum_{k=0}^{m-1}Po(k; ct)=e^{-ct}\sum_{k=0}^{m-1}\frac{(ct)^k}{k!}

And therefore, if we differentiate this, we will find that the density function is:

\displaystyle f(t) = ce^{-ct}\frac{(ct)^{m-1}}{(m-1)!}

And therefore, T\sim Ga(1/c, m).

Exercises

1. Find the constant c such that \displaystyle f(x) = \frac{c}{x^2+1} is a probability density and calculate the corresponding probability distribution function (Cauchy distribution).
2. From the density function of the Un(a.b), distribution, determine its corresponding distribution function.
3. Prove that the function \Phi_{\mu,\sigma}(x) is a probability distribution function.