Random Variables and Probability Distributions
Summary
This class provides an in-depth immersion into the concepts of random variables and probability distributions, fundamental pillars of probability theory and statistical analysis. The definition of a random variable as a number that depends on the outcome of a random experiment is introduced. The distribution function of a random variable is addressed, highlighting its importance as well as its essential properties. Finally, the relationship between random variables and probability distributions is analyzed, explaining that two variables can have the same distribution without being the same random variable.
LEARNING OBJECTIVES:
By the end of this class, the student will be able to:
- Understand the concept of random variables: Students should be able to describe and explain what random variables are and how they are mathematically defined.
- Understand the concept of probability distributions: Students should be able to explain what probability distributions are and how they are represented.
- Describe the properties of probability distributions: Students should be able to recognize and explain the key properties of probability distributions.
- Analyze the relationship between random variables and probability distributions: Students should be able to discuss how random variables and probability distributions are interrelated and how two variables can have the same distribution without being the same random variable.
- Demonstrate and apply the properties of probability distributions in practical situations: Students should be able to mathematically demonstrate the properties of probability distributions and apply these properties in real-world situations.
- Understand the concept of distribution functions: Students should be able to describe what a distribution function is and how it is used to describe a random variable.
CONTENTS INDEX:
What are random variables?
What are probability distributions?
Properties of probability distributions
Relationship between random variables and probability distributions
One of the key concepts of probability theory and statistical analysis is that of random variables and probability distributions. Although the theory we have developed so far is in a sense “complete,” the truth is that in its current state it is quite rudimentary; random variables and probability distributions are, so to speak, concepts that allow us to “oil our ability to work with probabilities and perform statistical analysis.”
What are random variables?
To familiarize ourselves with the concept of a random variable, it is useful to start with an intuitive approach: a random variable can be interpreted as “a number that depends on the outcome of a random experiment.” However, for a more precise understanding, it is essential to also explore its formal definition. Let’s see this definition:
Definition: A random variable over a set \mathcal{X} is a function f:\Omega \longmapsto \mathcal{X} |
The most common case is when \mathcal{X}= \mathbb{R}, and unless otherwise specified, this is what we will assume from now on; that is, we will work with real-valued random variables. Generally, random variables are denoted by uppercase letters, such as X,Y,Z, \cdots, while constants are denoted by lowercase letters. For simplicity, we will refer to random variables simply as “variables.”
Example: Suppose a 6-sided die is rolled twice. Then we have: \Omega_{2d6} = \{(\omega_1, \omega_2)\;|\; \omega_1,\omega_2 \in \{1,2,3,4,5,6\}\} From this, we can define the following random variables:
|
What are probability distributions?
Definition: A distribution function (or “DF”) of a random variable X is a function F_X: \mathbb{R} \longmapsto \mathbb{R} defined by the relation F_X(x) = P(\{\omega \;|\; X(\omega)\leq x\}), or more briefly: P(X\leq x). |
Generally, what interests us about a random variable is not so much its explicit expression in a sample space \Omega, but its distribution function. The subscript X in F_X can be omitted if the context is clear and there is no ambiguity. It is common to use the notation X\sim F to indicate that the random variable X has a distribution function F.
Properties of probability distributions
If F is a probability distribution and a,b are any real numbers, then the following properties will hold:
(a) a\lt b \longrightarrow [P(a\lt X \leq b) = F(b) - F(a)]
(b) a\lt b \longrightarrow F(a) \leq F(b), i.e., “F is increasing.”
(c) \displaystyle\lim_{x\to +\infty} F(x) = 1 and \displaystyle\lim_{x\to -\infty} F(x) = 0
(d) \displaystyle P(X=x)=\lim_{t\to x^+}F(t) - \lim_{t\to x^-}F(t)
(e) \displaystyle F(x)=\lim_{t\to x^+}F(t)
| PROOF (a) Let A and B be the events \{X\leq a\} and \{X\leq b\} respectively, with a\lt b. If all this happens, then we have A\subseteq B and therefore: \color{blue}{P(a\lt X\leq b)} = P(B\setminus A) = P(B) - P(B\cap A) = P(B)-P(A) =\color{blue}{F(b) - F(a)} (b) From part (a) we have: Since P(B\setminus A)\geq 0, we have: F(b) - F(a) \geq 0 which is the same as saying: F(a) \leq F(b) (c) Here we will use the fact that F is monotone increasing (proven in (b)) and bounded with a maximum value equal to “1” (because the distribution is defined in terms of probability). This alone is enough to say that: \displaystyle \lim_{x\to +\infty} F(x) = 1 A complementary approach allows us to make the following calculations with the same result. Let’s define the set A_n=\{\omega\;|\;X(\omega)\leq n\}. From this, it is easy to verify that for any n we have A_{n}\subseteq A_{n+1}, \displaystyle\bigcup_{n\lt +\infty} A_n = \Omega and therefore, using the property of continuity, we have: \displaystyle 1=P(\Omega) = P\left( \bigcup_{n\lt +\infty} A_n \right) = \lim_{n\to +\infty} P(A_n) = \lim_{n\to +\infty} P(\{\omega\;|\;X(\omega)\leq n\}) = \lim_{n\to +\infty} P(X\leq n)=\lim_{n\to +\infty}F(n) That is: \displaystyle \color{blue}{\lim_{x\to +\infty} F(x) = 1} Conversely, for the limit as x\to -\infty, we have the following: First, let’s define the set B_n=\{\omega\;|\;-n\lt X(\omega)\}. From this, it is verified that: \displaystyle \lim_{n \to -\infty}F(n) = \lim_{n\to -\infty} P(X\leq n) = \lim_{n\to \infty} P(X\leq -n)= 1 - \lim_{n\to \infty} P(-n \lt X) = 1 - \lim_{n\to \infty}P(B_n)) = 1 - P(\Omega) = 1-1=0 (d) The reasoning is similar to part (c). Let’s start by defining the set: \displaystyle C_n = \left\{x - \frac{1}{n} \leq X \leq x + \frac{1}{n}\right\} And from this, we have: C_{n+1}\subseteq C_n \displaystyle \bigcap_{n\gt 0} C_n = \{X=x\} Therefore, using a result from the continuity property, we have: \displaystyle P(X=x)=P\left(\bigcap_{n\gt 0} C_n \right) = \lim_{n\to \infty} P(C_n) = \lim_{x+1/n \to x^+}F\left(x+1/n\right) - \lim_{x-1/n \to x^-}F\left(x-1/n\right)= \lim_{t \to x^+}F\left(t\right) - \lim_{t \to x^-}F\left(t\right) (e) This last case is obtained from the previous result. In fact, since we have already proved: \displaystyle P(X=x)= \lim_{t \to x^+}F\left(t\right) - \lim_{t \to x^-}F\left(t\right) We can write: \displaystyle \lim_{t \to x^+}F\left(t\right) = P(X=x) + \lim_{t \to x^-}F\left(t\right) = P(X=x) + \lim_{t\to x^-}P(X\leq t)= P(X\leq x) = F(x) |
Relationship between random variables and probability distributions
Two variables X and Y are said to have the same probability distribution if (\forall A\subseteq \mathbb{R})(P(X\in A) = P(Y\in A)).
Two variables X and Y defined on the same sample space \Omega can have the same distribution but not necessarily be the same random variable. For example, if we consider the experiment of flipping a fair two-sided coin and X=1 corresponds to heads and X=0 corresponds to tails, we can define the random variable Y=1-X and have P(X=1) = P(Y=1)=0.5, and both have the same distribution, but if we calculate the probability that both have the same value, we have P(X=Y)=0