Probability/Important Distributions

Uniform Distribution

The uniform distribution is a model for "no preference". For example if we want to pick a positive integer between 1 and 10 with equal probabilities, the probability of picking each number would be 1/10. For the continuous analogon of such a distribution, there is an equal probability of picking any range of a given size. For example for a uniform distribution from 0 to 10, the probability of picking a real number between 1 and 2 is 1/10, the same as for picking a real number between 5.43 and 6.43.

This simple distribution forms the basis of much of the study of probability. It mirrors the type of activity observed from rolling a fair dice, or flipping a coin. This intuitively relates to many standard problems. A number of other distributions derive from this distribution.

The uniform distribution on the interval [a,b] is a continuous distribution with probability density f given by:

$f(x)=\{\begin{array}{cc}\frac{1}{b-a}& \text{for }a<x<b,\\ \\ 0& \mathrm{f}\mathrm{o}\mathrm{r}x<a\mathrm{o}\mathrm{r}x>b,\\ \\ \mathrm{s}\mathrm{e}\mathrm{e}\mathrm{b}\mathrm{e}\mathrm{l}\mathrm{o}\mathrm{w}& \text{for }x=a\text{ or }x=b.\end{array}$

The value of the density at a or b can be 0, $\frac{1}{b-a}$, or any other finite value without changing finite probabilities. For a more complete discussion of this point and for more information on the uniform distribution, see the Uniform distribution (continuous) page at Wikipedia.

A random variable X with this distribution is called a random number between a and b.

Binomial Distribution

Many random experiments are of the type where one counts the number of successes in a series of a fixed number of independently repeated trials which may result in either success or failure. The distribution of the number of successes is a binomial distribution. It is a discrete probability distribution with two parameters, traditionnaly indicated by n, the number of trials, and p, the probability of success.

A well known example of such an experiment is the repeated tossing of a coin and counting the number of times "heads" comes up.

One says that the random variable X follows the binomial distribution with parameters n and p, and writes X ~ B(n, p) if the probability of getting exactly k successes is given by the probability mass function:

$p(k)=(\genfrac{}{}{0ex}{}{n}{k})p^k(1-p{)}^{n-k}$

for $k=0,1,2,\dots ,n$ and where

$(\genfrac{}{}{0ex}{}{n}{k})=\frac{n!}{k!(n-k)!}$

is the binomial coefficient.

The formula is easily derived from the fact that from n trials k should be successfull, having probability $p^k$ and n − k failures, with probability $(1-p{)}^{n-k}$. This gives a specific outcome a probibility of $p^k(1-p{)}^{n-k}$ An outcome with exactly k successes, however, may occur in a number of specific orders, i.e. one has to choose k of the n outcomes to be successfull. This leads to the binomial coefficient as factor to be multiplied.

For more information see the binomial distribution article at Wikipedia.

The Poisson Distribution

The Poisson distribution with parameter $\lambda >0$ is a discrete probability distribution with mass function given by:

$p(k)=\frac{{\lambda }^k}{k!}{e}^{-\lambda }$

where k is a nonnegative integer.

The Poisson distribution is sometimes called: "the distribution of rare events", meaning it indicates the distribution of the number of events occuring in a fixed time interval, or fixed spatial area, where the occurence of an event is "rare" in the sense that no two events may happen together.

The random variable X indicating this number of occurences is called a Poisson random variable. The Poisson random variable is of fundamental importance in communication and queuing theory.

It is possible to obtain the Poisson distribution as the limit of a sequence of binomial distributions.

Let $\lambda$ be fixed and consider the binomial distributions with parameters n and $\lambda /n$:

$p_n(k)=\frac{n!}{k!(n-k)!}{\left(\frac{\lambda }{n}\right)}^k{(1-\frac{\lambda }{n})}^{n-k}=\frac{n!}{n^k(n-k)!}\frac{{\lambda }^k}{k!}{(1-\frac{\lambda }{n})}^{n-k}.$

In the limit, as n approaches infinity, the binomial probabilities tend to a Poisson probability with parameter λ:

$\underset{n\to \mathrm{\infty }}{\lim }{p}_n(k)=\underset{n\to \mathrm{\infty }}{\lim }\frac{n!}{n^k(n-k)!}\frac{{\lambda }^k}{k!}{(1-\frac{\lambda }{n})}^{n-k}=\frac{{\lambda }^k}{k!}{e}^{-\lambda }.$

This result shows that the Poisson PMF can be used to approximate the PMF of a binomial distribution. Suppose that Y is a binomial random variable with parameters n and p. If n is large and p is small, the probability that Y equals k is not easily calculated, but may be approximated by a Poisson probability with parameter np:

$P(Y=k)=(\genfrac{}{}{0ex}{}{n}{k})p^k{(1-p)}^{n-k}\approx \frac{(np{)}^k}{k!}{e}^{-np},$.

The above mentioned limiting process may be illustrated with the following example. Count the yearly number X of car accidents on an important roundabout. Let us assume on the average 8 accidents happen yearly. To calculate the distribution, we consider the 12 months and count a succes when an accident has happend that month and else a failure. The number of successes will constitute a binomial random variable $X_{12}$ with parameters n=12 and p=8/12. As the average number is 8, it is still likely some months will meet more than one accident, so we consider weeks and count the number of weeks with an accident as successfull. This number $X_{52}$ may be considered binomial distributed with parameters n=52 and p=8/52. While there may still be weeks with two accidents, we turn to days. The number $X_{365}$ of days with success may be considered binomial distributed with parameters n=365 and p=8/365. The limiting distribution by considering hours, minutes, seconds, etc. gives the Poisson distribution of X with parameter 8. We also note an important condition on the occurence of the events (accidents): we only count them correctly if in the end we may separate them in disjoint time intervals. So it is not allowed that two accidents happen the same moment.

Normal Distribution

The normal or Gaussian distribution is a thing of beauty, appearing in many places in nature. This probably is a result of the normal distribution resulting from the law of large numbers, by which a sum of many random variables (with finite variance) becomes a normally distributed random variable according to the central limit theorem.

Also known as the bell curve, the normal distribution has been applied to many social situations, but it should be noted that its applicability is generally related to how well or how poorly the situation satisfies the property mentioned above, whereby many finitely varying, random inputs result in a normal output.

The formula for the density f of a normal distribution with mean $\mu$ and standard deviation $\sigma$ is

$f(x)=\frac{1}{\sigma \sqrt{2\pi }}{e}^{-\frac{1}{2}{\left(\frac{x-\mu }{\sigma }\right)}^2}$.

A rather thorough article in Wikipedia could be summarized to extend the usefulness of this book: Normal distribution from Wikipedia.