This post is a repeat of a post from 2005

Operational definition is a term coined by Nobel Prize winning physicist Percy Bridgeman roughly 70 years ago. It is still not widely known or understood, yet it provides a backbone for understanding processes that is more powerful than almost any other factor.

Operational definitions have been called, “…definitions you can do business with…” Let us, for the sake of discussion, examine an operating instruction that says, before beginning the procedure, the table must be dry. That begs the question of what we mean when we say dry. The concept of ‘dryness’ is ephemeral. For the purposes of following the procedure, it must be defined. In and of itself to say, “the table must be dry”, means nothing. How dry? Very dry? Extremely dry? You see, more adverbs don’t help, we still have to say what dry means.

The definition will depend on the purpose. For example, in a restaurant one may say that “before setting the table, the busboy should make sure that it’s dry”. In this case ‘dry’ probably means no visible moisture. In the another business heating may be required to achieve the required level of dryness. How dry is defined depends on situation. The same is true for clean, safe, sterile, fast, religious, fit, very fit, alert and even logical. In order to be useful for measurement these terms must be defined. Is this table dry (enough)? It must be spelled out. If it is not two parties will disagree because they are using the same term in different ways.

This is particularly important in training. For example an instruction may be to fill the sink until it is full. What does this mean? How full is full? More importantly how would a person trying to do this job know when he or she had done it? People will guess. They won’t tell you they are guessing, but that is what will happen and two different people will arrive at two different definitions of when that job is done. Perhaps now one can begin to see the pervasive need for operational defintions. And, we have yet to even leave the workplace. What about customer and supplier. The customer says that on arrival the purchased goods must be clean. What does that mean? No visible soiling? Clean to the naked eye or clean at 100x? In order to be called a cotton shirt, the fabric can have no less than 90% cotton. What does this mean. Does it mean 100% cotton except for the collar? What does it mean to say a bill must be paid on time?

If we want to study the impact of various sales terms (e.g. Net Due, 1, 10% Net 30) on Whether or not bills are paid on time, we must have the same definition of ‘on time’ for comparison of the various sales terms. Otherwise differences that may appear to be due to the different sales terms could, in fact, be the function of differing definitions of ‘on time’.

Is this lack of operational defintions a problem? Without doubt it is and it is everywhere. As Deming used to say, “People on the job that don’t know what the job is.” If management is prediction and the enemy of prediction is variation then we have within our grasp a method of reducing variation and improving prediction simply by taking the time to operationally define our key concepts.

Oh I’m sure nobody thought this at the times the question is a very fundamental one.

It is fundamental because we live in a world of constant change. There’s nothing new about this realization. It’s been known for centuries. Heraclitus who lived in 500 BC said you cannot step out the same stream you step into.Both you and the stream had changed.

We live in a sea of variability. We are surrounded by it. We are immersed in. At one level we know this and accept it, but at another level we try to behave as though it weren’t true. This leads to problems, of course. Different traditions try to deal with this uncertainty in different ways. Some try to impose certainty.

In Buddhism this variability is called impermanence and much of Buddhism is intended to teach us how to live in the world of impermanence. Don’t cling to the past we are told. Don’t try to live in the future. It is only in the present moment that we encounter reality. Actually just another way to try to impose certainty. Eliminate variability with the wonderful idea of now.

But where does this leave us in our request to make order of the world.? How can we plan? How do we make decisions? How can we manage our lives?We are stuck making decisions in an uncertain world. We have no choice.

What is needed is a way to reduce uncertainty. A way, as the Young Rascals seek, to be sure.

At the very fundamental level, this dilemma is almost laughably simple. We wish to cross the street. We look at the traffic patterns, make predictions about the course and speed of the traffic and step off the curb and cross. We make decisions of this kind virtually every minute of every day. And, for the most part, life works well.

A more useful way to think about these decisions, any decisions really, is that they are predictions. We decide what we decide based on what we think will happen after the decision is made. That is, in the future. Making statements about what we think will happen in the future is prediction, by definition.

But, consider this. If everything we do is characterized by variability there is no certainty associated with any prediction. Variability is a rule of nature. It is unavoidable. Variable results are an inevitable consequence of any repetitious activity.

I don’t really think there is any debate about this. We see every day and all around us the variation inherent in existence.

Fortunately we have some tools with which we can navigate this morass. The tools are logic and statistical thinking.

That’s the good news. The tools are there. The bad news is that most of you have never learned them.

This is an interesting graphic that caught my eye. It is said to describe the amount of time, at average hourly wage for the worker’s country, a worker would have to work to be able to buy a Big Mac.

The good news is that I just moved and in the process came across a whole bunch of publications, including many by Deming, that I will proceed to read, comment on and extract what I feel are the salient points.

So stay tuned….

“Bright blue eyes have long been viewed as a quality that can help to attract the opposite sex. Now research suggests having piercing blue eyes may also say something positive about your level of intelligence.

Scientists believe they may have the explanation for why the likes of Stephen Hawking, Alexander Fleming, Marie Curie, and now Lily Cole are such brilliant academics – and it’s all in the eyes.

American researchers have found that blue-eyed people are more studious and are able to concentrate harder and outperform brown-eyed individuals in exams.

They are also likely to be more strategic thinkers who are able to plan their own time enabling them to work more effectively.

In return, those with brown eyes are likelier to be faster at running enabling them to be more likely to succeed in sports such as football, hockey and rugby.

The study, carried out by scientists at the University of Louisville, in Kentucky, could explain why so many academic geniuses have blue eyes.

Stephen Hawking, one of the world’s most eminent physicists and author of A Brief History of Time, has bright blue eyes as did Alexander Fleming, the biologist who discovered penicillin, and Marie Curie, who was twice given a Nobel prize for her pioneering work in radioactivity.

Lily Cole, the catwalk model whose features are dominated by her piercing blue eyes, recently secured a place at Kings College, Cambridge to read social and political sciences having gained five As at A level.

Another prominent blue-eyed mastermind is Stephen Fry, the author, actor and TV presenter, who gained a scholarship to Cambridge University.

The research may also explain why many of our top sportsmen such as John Terry, the England football captain, and Jonny Wilkinson, the former England rugby captain, are brown-eyed.

The scientists say that there is no logical explanation for the link between eye colour and academic achievement, but it is an unexplored area of research.”

To say that this article is riddled with foolishness is too kind. No wonder the public has so much skepticism about what they read about ‘scientific studies’ in the press. Their wariness is well justified in this case.

“Iran’s Guardian Council has admitted that the number of votes collected in 50 cities surpass the number of those eligible to cast ballot in those areas.

“The council’s Spokesman Abbas-Ali Kadkhodaei, who was speaking on the Islamic Republic of Iran Broadcasting (IRIB) Channel 2 on Sunday, made the remarks in response to complaints filed by Mohsen Rezaei — a defeated candidate in the June 12 Presidential election.

” ‘Statistics provided by Mohsen Rezaei in which he claims more than 100% of those eligible have cast their ballot in 170 cities are not accurate — the incident has happened in only 50 cities,’ Kadkhodaei said.

“The spokesman, however, said that although the vote tally affected by such an irregularity is over 3 million, “it has yet to be determined whether the amount is decisive in the election results,” reported Khabaronline.

The KEY phrase to the refutation being that the amount of votes counted exceeded the number of voters in only 50 cities….not the 170 originally reported.

Ok, but…..

Most award, reward, recognition and similar programs are aimed at the individual (treating every person as a special cause). Sometimes that’s appropriate, but generally it is the system as a whole (of which the people are a part) that produces the variation in performance. What did Deming say, over 90%?

If we apply Shewhart’s recommendation to this situation, we wish to reduce variability of the whole system and move the whole system average in the direction of goodness. Thus, what is needed, is action on the system as whole. Acting on individuals (Employee of the Month) doesn’t get that job done and may (if one thinks of tampering) make things worse.

This is not to say that people aren’t different. They are. Also this should not be taken to mean that individual differences don’t matter. They may be critical in a given situation. The point is that fixing the system requires action on the system, not the individual parts.

So what is needed is to create a system where people can grow.

Whether or not the test that was designed had sufficient discriminatory power is a matter for firefighting experts to decide. One group (the white firefighters) thought that it did and the other group (the non-white firefighters did not). As evidence that there was a bias built into the test the non-white group pointed out that the pass/fail ratio of white to non-white did not match the white to non-white ratio of test takers. This is a simple binomial probability problem (you set the alpha limit). I don’t know if it was examined using statistical methods, but it is a legitimate statistical question. (e.g. if the ratio of white to non-white test takers was (say) 60% to 40% of the total, what is the chance that out of 100 people taking the test 85 would be white and 15 non-white, IF the test were non biased (that is, if the null hypothesis was true).

But that was not what the Supreme Court case ended up being about. Apparently the bias was evident to some (probably non-white) test takers who threatened to sue the city on the basis that the test was unfair. Rather than reopen the original design of the test and fight the expensive lawsuit battle, the city opted to throw out the test and not use the results to promote anyone. Nobody (white or non-white) was promoted using the test.

The white firefighters sued the city saying that they had been discriminated against because they WOULD have been promoted had the test been used and the city could not simply disregard the test because of the bias issue or the threatened suit.

In a 5 to 4 decision the court agreed with the white firefighters group that they had been discriminated against by the city’s actions. It is a tricky case in the sense that since no one was promoted using the case, no individual can show that he or she should have been selected over another individual. Also it adds a burden to cities to stick by testing protocols even when they change their mind (e.g. in the face of a lawsuit).

If you are interested read the opinion and Bader-Ginsberg’s dissent see:

http://www.supremecourtus.gov/opinions/08pdf/07-1428.pdf

One blogger on the GOP side even published some data (gasp!). the correlation matrix she published here is presented below together with her conclusions.

Singer’s blog presents a wonderful case of obscure statistics which she then goes on to completely ignore when they show a result with which she doesn’t agree. Why publish it in the first place? Obviously to give her piece, which is basically innuendo, the aura of scientific credibility.

Her conclusion number one:

“Why would there be a highly positive correlation between dealer survival and Clinton donors?”

There isn’t. The correlation is .59 with a sample size of 53. There is no correlation there at all much less a “highly positive” one.

Her conclusion number two:

“Nevertheless, it seems clear that something is going on here.”

Yes there is something going on, but it has nothing to do with this data. What is going on is political innuendo and a ‘guilt by association’ attack. Nothing new about that in politics, but throwing these data around (which few will read much less understand) is a giant smoke screen to mask the real intent.

But since, as we have discussed, samples of one can be extremely misleading, shouldn’t newspaper accounts using them be disregarded? Well, to be the real statistically minded person I am, I must say, “It depends”.

The question itself poses a larger issue, however and when considered in full, much of what is reported in news accounts may be distorted or contain (or lead to) totally unwarranted conclusions. Reporters, based on my sample of two, are reluctant to change. The first person account has literary merit and is a good way to lead into an article from the point of view of grabbing the readers’ attention.

The next time you read a news article on the internet, in the paper or in a magazine note the extent to which the conclusions of the article or its summary are based on stories of individuals or small groups. Ask yourself if a different group or person had been selected, the extent to which the conclusions might be different. Sure people get mugged in big cities and their stories are compelling, but the fact is, that even in the most dangerous cities, most people don’t get mugged.

In the same way that no two widgets produced by the widget machine are the same, no two experiences are the same among individuals. Different people experiencing the same event can have vastly different reactions to it. In fact, they are not even experiencing the same event. Of what use then is an eyewitness account? Such person accounts hinder generalization because they cannot, in themselves, show the range of experience that many individuals do have. In that way they are misleading.

Another problem with relying on first person accounts as evidence upon which to base a conclusion is that they are very unreliable. They are based not only on interpretation of an event at some moment in time, but as they are related to the reporter they are a story that is a memory of that event. Like all stories these accounts change over time and may eventually stray far from the actual event as it unfolded.

Advertising often uses anecdotal evidence. After all, that’s really what a testimonial is. Assuming that the spokesperson has even used the product (which is sometimes pretty clearly not the case), it is still a sample of one person’s opinion. Interesting, but no basis at all for generalization.

Please don’t misunderstand. Opinions are great. But one person’s opinion is only a good indicator of what happened if there was no variation and, if we know nothing else, we know that life is nothing but variation.
For example a new reporter writes the story of a family’s experience as they struggle through economic hard times. The story may be compelling. Indeed a good writer’s job is to make it so. But, what does the story of that family tell use about what families across America are experiencing in their economic struggles? Not very much.

The breadth of experience of citizens as they struggle economically is enormous. No two familial situations will be the same and the range of experiences can be huge. The family cited in the news account only gives us one data point. To truly understand the economic picture in the country we need dozens of data points, perhaps hundreds. Maybe they should be stratified by region or industry and so on.

Samples of one are not always labeled as such, but they are everywhere. The message of statistical thinking is to be extremely leery of drawing conclusions or making generalizations based on such a small amount of evidence.