Russel L Ackoff & Fred E. Emery

Errors of Observation

(1972)

 



Note

An interesting text on the possible pitfalls in observation.

It is taken from On Purposeful Systems, Appendix I, Tavistock Publications.

 


 

There are four possible sources of error in observation:

(1) the observer himself,
(2) the observed,
(3) the instruments used in making observations, and
(4) the environment in which the observations are made.

Furthermore three possible types of error can be produced by these sources:
(a) observing inaccurately (as miscounting or mismeasuring),
(b) not seeing something that is there, and
(c) seeing something that is not there.

Because of these errors we consider some people to be better observers than others, and several tests have been developed for evaluation of observers.

Kirk and Talbot (1966) have named these three types of observational error as
(a) systematic or stretch distortion,
(b) fog distortion, and
(c) mirage.

Each of these types of error can be produced by any of the four sources of error. (See table)

Sources and types of errors of observation

Sources of Error
Types of Error
  Systematic Fog Mirage
1 Observer      
2. Observed      
3. Instruments      
4. Environment      

 

Systematic Distortion

In SD [systematic distortion] no information is lost. Rather, it is changed or recorded in an orderly or systematic way. Distortions of this kind are like the distortions a rubber sheet might undergo, so long as it is not torn. Thus, SD can be eliminated or "corrected for" by the application of a rule specifying the appropriate "topological transformation" (p. 310).
Kirk and Talbot cite the following example of systematic distortion produced by an observer.

Astronomer Maskelyne fired his assistant, Kinnebrooke, because the latter was clearly incompetent. Charged with clocking upper transists of certain reference stars, Kinnebrooke consistently clocked them "late" (p. 308).

They illustrate instrument-produced systematic distortion as follows.

Some auto rear-view mirrors are cylindrically convex so that a driver may scan at a glance far more than a "flat-mirror glimpse" of the territory behind him. Again, he sees images which are tall and thin, and they require "getting used to."

A bathroom scale that is improperly set will also produce a systematic bias in readings of persons' weights. An example of observed-produced systematic distortion is found in a subject who, on being interviewed, always, or almost always lies. If he always lied we could easily correct for this distortion by attaching a "not" to his main verbs. Environment-produced systematic distortion is introduced, say, by a nonwhite light when we are trying to determine the color of objects. Changing temperatures will also change the length of metal bars and hence may produce distorted observations. These could be corrected if we know the temperature and the coefficients of linear expansion of the metals under observation.


Fog

Fog occurs when an observer does not see what is there. In such distortion ". .. information is lost, mashed out, 'fogged' over ..." (p. 313). An observer may not be able to hear sounds above an abnormally low frequency or volume if he is partially deaf. If he is color blind, then, of course, he fails to observe color. Recording equipment may also fail to pick up low-volume sounds or high frequencies. Film may fail to capture color. (If it distorts color, the result is systematic distortion, not fog).

Noise in an environment may result in our failure to hear certain sounds. Glare may prevent our seeing objects that would otherwise be clearly visible. A subject in an interview who lapses into a language or use of words that we do not understand introduces fog into the exchange. Ambiguity is a type of fog. For example, some feel that James Joyce produced an inpenetrable verbal fog in Finnegan' s Wake.


Mirage

"In mirage distortion (MD) we see something that 'isn't there.' Far from withholding information from us, MD gives us extra, unwanted information" (p. 316).

Most of us have seen or heard things that were not there or tasted ingredients in food that were not there. A subject in an interview can deliberately (or not) produce in us a belief of the occurrence of an event that never took place. A burglar alarm system may go off because of an internal defect when no intruder is present. A false alarm is a mirage. In a very noisy environment we may hear things that were not said.

Hence there are four sources and three types of observational error. Implicitly or explicitly each observer has relevant beliefs with respect to each, and these determine whether or not use will be made of the data obtained. When the observer believes that error is present, he may be able to correct for it if he knows its source and nature. For example, he can correct for the bias of the bathroom scale or the late response of another observer. By interpolation he can fill in missing data, and by a wide variety of tests he can eliminate inconsistent data. The theory of data adjustment is frequently used in science for just this purpose (Deming, 1943).

 


 

References

Deming, W. E. Statistical Adjustment of Data. New York: John Wiley & Son, 1943

Kirk, J. R., and G. D. Talbot. "The Distortion of Information." In Communication and Culture, edited by A. G. Smith, pp. 308-21. New York: Holt, Rinehart & Winston, 1966.

 


[Home] [Top]