Statistics/Point Estimates

Point Estimates

Definition: Suppose a random variable $X$ follows a statistical distribution or a law ${ℒ}_{\theta }$ indexed by parameter $\theta$. Then a function $g:X\to \hat{\theta }$ from the sample space to the parameter space is called a point estimator of $\theta$.

In general, let $f(\theta )$ be any function of $\theta$. Then any function from the sample space to the domain of $f$ will be called a point estimator of $f(\theta )$.

Definition: If $h$ is a point estimator for $\theta$, then for a realization $x$ of the random variable $X$, the quantity $h(x)$ is called a point estimate of $\theta$ and is denoted as $\hat{\theta }$.

Notice that the estimate $\hat{\theta }$ is a random variable (unlike the true parameter $\theta$), since it depends on $X$.

Examples

1. Suppose $X_1,X_2,...,X_n$ follow independent Normal(μ,σ²). Then an estimator for the mean μ is the sample mean $\frac{\sum X_i}{n}$.
2. Suppose $X_1,X_2,...,X_n$ follow Uniform[θ,θ+1]. Then an estimator for θ is $\min X_i$. Another is $\max X_i-1$. Yet another is $\frac{\sum X_i}{n}-\frac{1}{2}$

Notice that the above definition does not restrict the point estimator to only the "good" ones. For example, according to the definition it is perfectly fine to estimate the mean $\mu$ in the above example as something absurd, like $10\sum X_i^2+\exp (X_1)$. This freedom is in the definition is deliberate. In general, however, when we form point estimators we take some measure of goodness into account. It should be kept in mind that the point estimators will always be targeted to be close to the parameter it estimates, intuitively and if possible, formally.