In this post I will explain the basics of the mathematical formulation of probability theory. Firstly, a probability space ties together the ideas of events and the probability of these events occurring. The exact definition and methods of determining these probabilities are disputed (i.e. Frequentist vs. Bayesian), however we will take these as given. A probability space is given by the triple: , where is the sample space, is an -algebra on subsets of , and is a measure which maps the elements of to the interval .
The sample space is the set of all possible outcomes for the system. For example, if we are examining the probability of rolling a die, the sample space is . The subsets of the sample space that contain only one element are said to be the atomic or elementary events. The -algebra over the set is the collection of subsets of with the following properties:
- , .
- (closed under complementation).
- where (closed under the union of countably many sets).
When discussing discrete probability spaces and finite sample spaces (which will usually be the case here), can be expressed generally as , the power set of . Any element of is called an event, which could also be thought of as the union of some elementary events.
Let be a function that defines the probability of each elementary event. In a discrete probability space we can define , the probability of an event is defined as .
Let us examine the example of the probability space of one roll of a six-sided die. Using the following definitions
If we are interested in the event that we roll an even number we can see:
.
