Elementary Probability - Computing with Events

• The probability that the first card is a club given that the second is a club.
• The probability that the first card is a diamond given that the second is a club.

First, we'll define a few terms:

• Experiment: An action that produces an outcome that we're interested in, and which can be, at least in thought, repeated under exactly the same conditions.
            e.g., Flip a coin

• An outcome is an observable result of an experiment.
            e.g., Heads

• The sample space for an experiment is the set of all possible outcomes.
            e.g., {heads, tails}

• Typically, we use Ω to represent the sample space.

Perhaps the most important term:

• An event is a subset of the sample space
            => Any subset is an event.

Correspondence between events and "word problems":

• Consider this problem: what is the probability of obtaining exactly 2 heads in three flips of a coin?
  • What is the sample space?
              Ω= {(H,H,H), (H,H,T), (H,T,H), (H,T,T), (T,H,H), (T,H,T), (T,T,H), (T,T,T)}
  • What is the event of interest?
              Let E = {(H,H,T), (H,T,H), (T,H,H)}

• Example: what is the probability of obtaining an odd number in a die roll?
  • What is the sample space?
              Ω= {1, 2, 3, 4, 5, 6}
  • What is the event of interest?
              Let E = {1, 3, 5}?

• The "occurrence" of an event:
  • Consider a die roll and the event E = {1, 3, 5}?
  • Suppose you roll the die and "3" shows up
              => Event E occurred (because one of its outcomes occurred).

Probability:

• More precisely, a probability measure.

• A probability measure on a sample space Ω is:
  • A collection of numbers, one number for each event.
              Notation: Pr[E] is the "number" associated with event E
  such that the following "rules" are satisfied:
  • (Axiom 1) Pr[Ω] = 1.
  • (Axiom 2) 0 ≤ Pr[E] ≤ 1 for any event E.
  • (Axiom 3) For disjoint events A and B, Pr[A union B] = Pr[A] + Pr[B]

• Alternative notation: Pr{E} (for event E).

• Example: Consider a single coin flip
  • Sample space Ω={H,T}
  • We'd have to specify a number for each possible event:
              Pr[{H}] = ?
              Pr[{T}] = ?
              Pr[Ω] = 1 (always 1, by definition)
              Pr[φ] = ? (empty subset)
  • For example:
              Pr[{H}] = 0.6
              Pr[{T}] = 0.4
              Pr[Ω] = 1
              Pr[φ] = 0 (must be zero)

• Note: we have specified the numbers as a (complete) list.
            One number for each event.

• How do we know the above measure is a valid measure (satisfies the axioms)?

  We can laboriously check:

  • Axiom 1: Pr[Ω] = 1
              => Satisfied.
  • Axiom 2: 0 ≤ Pr[E] ≤ 1 for any event E.
              => Satisfied for each event in list.
  • Axiom 3: For disjoint events A and B, Pr[A union B] = Pr[A] + Pr[B]
              => We need to check for each possible combination of disjoint events.
              e.g., A={H}, B={T}: Pr[A union B] = Pr{H,T} = Pr[Ω] = 1 = 0.6 + 0.4 = Pr[A] + Pr[B]

Operations on events:

• The usual set operators apply to events, since events are sets.

• A' = Ω-A = complement of A.

• A and B = intersection of event A and event B.

Other ways of specifying probability measures:

• For a die-roll, specify as follows:
            Pr[E] = |E| / 6, for any event E.

• Note: this achieves two purposes
  • Compact representation (instead of full list of events).
  • Assumes Axiom 3 is satisfied
              => Axiom 3 is used by construction
              e.g., Pr{1,3,5} = Pr{1} + Pr{3} + Pr{5} = 1/6 + 1/6 + 1/6 = 3/6

• Similarly, for a single card drawing:
            Pr[E] = |E| / 52, for any event E.
  • For example, E={0,1,...,12}
  • Pr[E] = 13/52    (using Axiom 3)

• Notation: we will occasionally use an informal event description (in English) e.g.,
            Pr[draw a spade] = 13/52.

Simple and complex experiments:

• Consider two die rolls:
  • Sample space: Ω = {(1,1), (1,2), (1,3), (1,4), (1,5), (1,6), (2,1), (2,2), (2,3), (2,4), (2,5), (2,6), (3,1), (3,2), (3,3), (3,4), (3,5), (3,6), (4,1), (4,2), (4,3), (4,4), (4,5), (4,6), (5,1), (5,2), (5,3), (5,4), (5,5), (5,6), (6,1), (6,2), (6,3), (6,4), (6,5), (6,6)}
  • Thus, there are 2³⁶ possible events.
  • One way to specify: Pr[E] = |E|/36, for any event E.
  • Example: Pr[same number on both dice] = 6/36.

• More complex experiments can be constructed out of simpler ones.

A minor tweak to Axiom 3:

• Since Pr[A union B] = Pr[A] + Pr[B], it's true that Pr[A union B union C] = Pr[A] + Pr[B] + Pr[C]

• In general, Pr[ union of A₁, ..., A_n] = Pr[A₁] + ... + Pr[A_n]

• But consider a sample space like this: Ω = {1, 2, 3, ...} (countably infinite)
  • Let event A_i={i}.
  • Let event A = union of all A_i's.

• Countable additivity axiom: Pr[A] = Pr[A₁] + Pr[A₂] + ...
            => Axiom 3 needs to hold for countable unions of disjoint events.

The meaning of probability: the frequentist interpretation

• Consider an experiment, an event E and the number Pr[E].

• Suppose we are able to repeat the experiment any number of times.

• Let f(n) = number of times event E occurs in n repetitions.

• Then,
            lim_n→infty f(n)/n = Pr[E].

Conditional probability

        ...
        double numBuses = 0;
        while (arrivalTime < myArrivalTime) {
            busStop.nextBus ();
            arrivalTime = busStop.getArrivalTime ();
            numBuses ++;
        }
        ...