Samples of one

A target shooter has a new rifle and wishes to adjust the sights so that the rifle is accurate. He loads the rifle and places a target some distance away and then assumes his firing position and fires a shot. He examines the target and finds that the hole produced by the shot is well to the left of the target center and below the bulls eye as well. What should he do?

Most people who give this any thought get the right answer. The shooter should fire another round, and then another…

After one shot the shooter has only one piece of information. Where the bullet struck. That provides no information as to the accuracy of the rifle. In a previous post, I mentioned the idea of ‘embracing the ontology of flux’. Once that conceptual leap is made, the individual recognizes that no two shots can ever be the same and that a myriad of factors goes into the variability of each shot. Some of these are: the steadiness of the shooters aim, the shooter’s visiual acuity, the ambient conditions of wind and weather, movement of the target, differences among bullets, and so on. Each of these varies a little bit (or sometimes a lot) from shot to shot producing the variability we see in the pattern of holes in the target. With one shot only, there is no pattern to be seen. There is no variability with a sample of one.

How many shots should the shooter take before adjusting the sights of his rifle? At least three or four. Consider the following patterns of four shots:

The rifle is not very accurate. Since the shots are scattered all about the bullseye, adjusting the sights won’t help. Get closer to the target.

The rifle is accurate and an adjustment to the sight would produce a tight cluster in the bulls eye.

Good news. Keep shooting.

It is evident that without multiple samples, patterns of variation cannot be seen and knowledge is meager. The predictions made without sufficient knowledge of variation are fraught with peril. Shewhart indicated that one of the key components of variation was some sort of knowledge basis. This is, in part, what he was talking about

Yet, we make this mistake all the time. It is called anecdotal evidence. Wishing to make some point or other, we produce a case that makes our point. But there is no variation in one case. It is a sample of one. There is no knowledge of what has happened in the same circumstances at a different time or different place.

The case study method of teaching is exactly this kind of meager approach to gaining knowledge.

It is a perilous course.