uncertainty

Uncertainty Analysis

by Matt Young

Based on a talk given to the Laser Measurements Short Course, Boulder, Colorado, August, 2000

No measurement is exact

Measuring stick is not calibrated precisely correctly
Repeated measurements of the same quantity yield slightly different values
Meter has only a finite number of digits
Constants themselves are known only approximately
Change of some external variable changes the outcome of the measurement

A measured value is therefore meaningless without some statement of its accuracy

Heresy

Define true value

Measurements deviate from true value
either randomly
or systematically
Deviations from the true value are called errors

Calculate uncertainty by analyzing errors

Postmodernism: No fact can be known with absolute certainty

GUM: True value does not exist; only measurements exist

No errors; only deviations from mean

I take the approach deprecated in Annex E.5 of GUM (Guide to the Expression of Uncertainty in Measurement; see References)

The result is the same as the GUM's but, uh, intelligible and more nearly consistent with common parlance

Errors

True value of measurand = X
N measurements x_i, i = 1, 2, ..., N

Mean value

µ = (1/N) S₁^N (x_i)

is an estimator of X

Measurements may or may not cluster about the true value

(1) The values x_i cluster about the true value X.

(a) The spread is narrow, so µ is a good approximation to X.

(b) The spread is wide, so µ is only a fair approximation to X.

(2) The values x_i do not cluster about the mean, so µ is not a very good approximation to X

Statistical errors

Error e_i is the difference

e_i = x_i - X

We do not (and cannot) know X

Approximate X by the mean µ of the experimental values:

e_i* = x_i - µ

e_i* approximates e_i and is called an estimator of e_i.

Errors that cluster about their mean (not necessarily the true value!) are called random errors
Sample standard deviation is an estimator of the average error

s = sqrt[(1/(N-1)) S₁^N (e_i*²)]

µ and s will vary from sample to sample

Gaussian or normal distribution of errors

Not all errors follow a Gaussian distribution (cosine error, for example)

Standard deviation of the mean

Peak of the Gaussian curve estimates the value for which e_i = 0 (not X = 0)

More data --> easier to locate peak of curve

Uncertainty of set of measurements not the width s of curve but rather

s = s/sqrt(N)

is the standard deviation of the mean
as opposed to the sample standard deviation s

Type A uncertainty

Uncertainty that is measured by statistical means

s is an estimator of the uncertainty of the mean of a set of measurements
Decrease s to small value by making large number N of measurements
Square root in the denominator diminishing returns

Mean µ of N measurements has a 68 % probability of being between µ - s and µ + s , or

68 % probability of being in error by less than
We define s, not s, as the standard uncertainty

u_r = s

due to random errors

Standard uncertainty is always positive

Experimental results are expressed as

mean ± k times standard uncertainty

Usually k = 2 (see below)

Type B uncertainty

Uncertainty that is not measured statistically

Measuring instruments not precisely calibrated
Drift in a meter
Constant offset, as due to background intensity
Uncertainty imported from manufacturer
Etc., etc., and so forth

I think of Type B errors as belonging to several subsets

None defined in the GUM

Estimated errors (my terminology)

Example

Measure long distance with steel tape

Estimate the error due to changes in the length due to thermal expansion
Estimate the range of temperatures
Coefficient of thermal expansion
Uncertainty of coefficient

Expected temperature range is ±DT
Maximum error of length = e_m (calculated)

Rectangular distribution

Assume any value between µ - e_m and µ + e_m is equally likely
Standard deviation of a rectangular distribution = e_m/sqrt(3)
Therefore, define standard uncertainty

u = e_m/sqrt(3)

Example

Digital voltmeter, no electronic noise

Least significant digit = 1 mV
e_m = 0.5 mV
Standard uncertainty = e_m/sqrt(3),
or 0.5 mV/sqrt(3) = 0.3 mV

Note no division by sqrt(N) since repeated measurements will be identical

Imported errors (my terminology)

Manufacturer specifies an uncertainty

Example

Uncertainty specified as 1 % of full scale
On 1 V scale, DV = 10 mV
1, 2, or 3 times s?
Assume that the manufacturer means 2s
Take e_m = 3s = 1.5 DV = 15 mV (1.5 times the manufacturer's quoted uncertainty)

More generally,

u = 1.5 u_f/sqrt(3)

(f for manufacturer)

Systematic errors (deprecated term)

Errors that generally have only one sign

either positive or negative
result in an offset or a bias

Example: thermal expansion in a steel tape

Could be + or - but at any one time is constant

Example adapted from micrometry:

Distance between two parallel but rough walls

Cosine error: measured value high by 1/cosq
No matter the sign of q
Estimate a mean value of q and correct for error
Estimate the uncertainty of correction

Example continued

Roughness of walls
Measurement = distance between the high points
Peak-to-valley distance = 1 mm
Assume roughness completely random
Mean position of each wall = 0.5 mm behind the peaks
Measuring rod contacts the peaks
Measurements too low by 0.5 mm, each wall
Correct bias by adding b = 0.5 mm, each wall

Uncertainty of correction

Assume maximum error e_mof the correction = one-half the correction itself

e_m = b/2 = 0.25 mm, each wall
Standard uncertainty u_g = 0.25 mm/sqrt(3) = 0.14 mm, each wall
Due to both walls, sqrt(2) u_g
More conservative? Assume e_m = b, not b/2

This approach allows us to specify a result ± a single uncertainty

rather than mean + u₁/-u₂

To sum it up:

Table 1. Standard uncertainties.
Type of uncertainty	Distribution of errors	Standard uncertainty	Value
Random or statistical (Type A)	Gaussian	Standard deviation of mean
Estimated (including uncertainty of systematic error or bias) (Type B)	Rectangular	Standard deviation of rectangular distribution	e_m/sqrt(3)
Imported (Type B)	Rectangular	Manufacturer's specification	1.5 u_m/sqrt(3)
Systematic (Type B)	Bias	--	b

Combined standard uncertainty

u_c = sqrt(u_r² + u₁² + u₂² + u₃² + ...)

Express experimental results in form

µ ± 2 u_c

U = 2u_c = expanded uncertainty
Factor 2 is called coverage factor

Coverage factor of 2 means
confidence interval is between µ - 2s and µ + 2s, or
95 % confidence interval

¡Uncertainty analysis is approximate and subjective!

Subjective estimates of many parameters
Arbitrary assumption of rectangular distribution
Assumption that uncertainties are uncorrelated
Ignoring of high-order terms

Dirty secret:

A statement of uncertainty tells what we think about our measurement more than it tells about the measurement itself (thanks to Ron Wittmann)

Appendix *

Older sources add systematic errors arithmetically. In our notation,

u_c = sqrt(u_r²) + u₁ + u₂ + u₃ + ...

where u_i here represents a systematic error

The logic was this:
All the systematic errors may well have the same sign
Add them for a conservative estimate

No longer accepted
No reason to believe that systematic errors will have same sign if they are not correlated

* A vestigial part of a book for which no one has yet discovered a use

References

Anonymous, Guide to the Expression of Uncertainty in Measurement, International Organization for Standardization, Geneva, 1993.

Barry N. Taylor and Chris E. Kuyatt, Guidelines for Evaluating and Expressing the Uncertainty of NIST Measurement Results, Natl. Inst. Stand. Technol. Tech. Note 1297, Washington, 1994. Available on the Web at http://physics.nist.gov/Pubs/guidelines/outline.html. This is sort of a guide to the GUM.

John Taylor An Introduction to Error Analysis: The Study of Uncertainties in Physical Measurements, University Science Books, Mill Valley, California, 1997. (This excellent book does not teach or conform to the methodology of the GUM.)