INDICES OF PRECISION
It is important to be able to assess the precision of a set of observations, and several standards exist for doing this. The most popular is standard deviation (σ), a numerical value indicating the amount of variation about a central value. In order to find out how precision is determined, one must first consider a measure which takes into account all the values in a set of data.
Such a measure is the deviation from the mean (¯x) of each observed value (xi), i.e. (xi − ¯x), and one obvious consideration would be the mean of these values. However, in a normal distribution the sum of the deviations would be zero because the sum of the positive deviations would equal the sum of the negative deviations. Thus the ‘mean’ of the squares of the deviations may be used, and this is called the variance (σ2).
Theoretically σ is obtained from an infinite number of variates known as the population. In practice, however, only a sample of variates is available and S is used as an unbiased estimator. Account is taken of the small number of variates in the sample by using (n −1) as the divisor, which is referred to in statistics as the Bessel correction; hence, variance is:
As the deviations are squared, the units in which variance is expressed will be the original units squared. To obtain an index of precision in the same units as the original data, therefore, the square root of the variance is used, and this is called standard deviation (S), thus:
Standard deviation is represented by the shaded area under the curve in Figure 2.5 and so establishes the limits of the error bound within which 68.3% of the values of the set should lie, i.e. seven out of a sample of ten.
Similarly, a measure of the precision of the mean (¯x) of the set is obtained using the standard error (S¯x), thus:
Standard error therefore indicates the limits of the error bound within which the ‘true’ value of the mean lies, with a 68.3% certainty of being correct.
It should be noted that S and S¯x are entirely different parameters. The value of S will not alter significantly with an increase in the number (n) of observations; the value of S¯x, however, will alter significantly as the number of observations increases. It is important therefore that to describe measured data both values should be used.