Statistical & Scientific Thinking

Interactions and Systems

John — Sat, 26 Jul 2008 16:08:57 +0000

This discussion goes to the heart of a subject which we don’t discuss nearly enough and that is interactions.

Hypothesis Testing III - The statistics

John — Sun, 20 Jul 2008 21:07:27 +0000

To continue, we need to define a couple of terms. The first is a probability density function and the second is a sampling distribution.A probability density function expresses a particular function in terms of integrals. Thus for a frequency distribution smoothed (over repeated sampling) to form a curve as shown below, the area under the curve can be calculated and the probability of a given value occurring in the distribution can be assessed as a proportion of the amount of the curve that is to the left and/or to the right of the value. In a normal distribution, z values are used to do this.

(more…)

Hypothesis Testing II - The logic

John — Mon, 07 Jul 2008 18:09:07 +0000

A hypothesis is a supposition made as a basis for research or reasoning without regard for its truth. So says the Oxford dictionary. What starts the hypothesis testing process is just such a supposition. In the justice system example we made, there is a supposition on the part of law enforcement that the apprehended person committed the crime.In pharmaceutical testing, there may be a supposition a given medicine will reduce cholesterol values in blood tests.Because of the nature of induction we can never prove theory going forward. As a prediction we recognize that the samples of interest are not those of the past, but rather those of the future and they have not happened yet. Thus they are not available to be sampled for our study and they may, in some way, be different from the samples that are available for the study.Further, that difference could render the results that are (necessarily) based on the samples in the study inapplicable to the future samples. This problem is unavoidable. So we design a way to test our hypothesis in another way. One aspect will be that the statements we can make will be probabilistic. We are in the land of uncertainty.To do our test we create another hypothesis called the null hypothesis. The Null hypothesis is essentially an assumption that the research supposition (Research Hypothesis) has no merit. In our justice system case, it is an assumption of innocence. In the pharmaceutical case the null hypothesis would state that the new drug had no effect on lowering cholesterol.So there are two hypotheses. The null hypothesis (usually designated H0 and the research hypothesis which is usually, and somewhat unfortunately, called the alternate hypothesis (usually designated H1 or HA). I say unfortunately because this is the hypothesis that we are most interested in and the word ‘alternate’ makes it seem almost secondary.Thus the comparison is constructed in such a way as to pose the null hypothesis and the burden of rejecting that (no difference) hypothesis rests with the advocate of the alternate hypothesis. That is, if the research can provide enough evidence to reject the null hypothesis, there is reason to believe that the new drug does have an affect on cholesterol. If the prosecutor can provide enough evidence we will suggest that the apprehended person did commit the crime.We could be wrong. People sometimes ask a statistician to give a yes or no answer or to provide certain proof. That simply cannot be done and that is a problem fundamental to the inductive nature of this process. It is for this reason that we do not accept the alternate hypothesis in the sense that it is ‘proven’. The logic of hypothesis testing this way leads to either reject the null (no difference) hypothesis or to not reject it.A standard of proof is given. In the justice example in the United States that standard is that the proof must be ‘beyond a reasonable doubt’. In a research study, such as the pharmaceutical example, we set probability level (more about this later).If that standard is met or exceeded we reject the null hypothesis. If the standard is not met we do not reject it.

Quality, Productivity and Competitive Position

John — Fri, 04 Jul 2008 20:44:06 +0000

June marked the 25th anniversary of the publication of Dr. W. E. Deming’s first management book, “On Quality, Productivity and Competitive Position”. Deming lived both in Washington D. C. and New York where he kept an apartment on Hudson St. in the Village. He was a professor at NYU’s Graduate School of Business Administration from 1946 until his death in 1993. He also taught for years at Columbia. While returning to his apartment one evening in 1968 he was mugged and stabbed and was rushed to St. Vincent’s Hospital where he spent several days; a true New Yorker, indeed.Deming was born in 1900 in Iowa and spent the bulk of his early childhood in Polk City and then at around age 7 moved to Cody, Wyoming and later Powell, Wyoming. Life was harsh and he lived, as did many, in a tarpaper shack. He would joke in his seminars that he would wager he was the only person in the room that was born in the reign of Queen Victoria.For those unfamiliar with Deming (which seems to include much of corporate America) he is perhaps best known for his trips to Japan after the end of World War II. He had traveled to Japan in 1947 originally to help occupation forces to conduct sampling research for a census.His visit was not unnoticed by the fledgling Union of Japanese Scientists and Engineers (JUSE) and he was invited to return to Japan at a later date to conduct a series of classes on the statistical quality control techniques that had proved so beneficial to the United States in the production of high quality materials for defense manufacturers.He did return in 1950 and conduct those classes. They were primarily in the area of what, at that time, was called Statistical Quality Control. But Deming also taught another course. The essence of this teaching is captured in the speech he gave to many of the top managers of Japanese post-war management at Mt. Hakone in 1950.At that meeting Deming outlined a competitive strategy that has come to be known as the “Deming Chain Reaction”. Basically it is simple in concept, but difficult in execution.It had always been thought that better quality could only be achieved at higher cost. That is, there is a theoretical point at which further attempts to improve quality would inflate price so much that the object being manufactured is no longer marketable because of its cost. As it turns out, that is only true if attempts to improve quality are focused on inspection. When one focuses attention on quality through process improvement and the elimination of waste, quality improves and costs go down at the same time.There is no better example of this strategy at work than Toyota Motors. But there are many, many other examples as well. We, in America, are all too familiar with the phenomenon of products invented and developed here that are manufactured elsewhere. The result has been a huge deindustrialization of America. Industrial America today is half the size it was 30 years ago. The effect of this deindustrialization on the economy has been devastating.Deming predicted Japan’s success in the 50s when no one else was paying attention to them economically at all. He foresaw the de-industrialization of America and entitled his next book, “Out of the Crisis”. Both books contain the basics of his ideas about how to be competitive. Perhaps for America he was too far ahead of his time.In keeping with his upbringing Deming worked right up until his death in 1993. He maintained his active consultancy and taught his famous “4-Day” seminar until a few weeks before the end. He was tough.I had the good fortune to know him personally and he was a wonderfully kind and gracious man as well. I can still clearly hear his words, “There is no substitute for knowledge.” The knowledge is there in the pages of his book. It needs only to be extracted and acted on.

Tests of Hypothesis

John — Thu, 03 Jul 2008 16:37:07 +0000

When discussing hypothesis testing, I have, from time to time, used the example of the criminal justice system to act as a metaphor for the logic and philosophical issues involved. It remains a good way to pose the logical dilemmas and the types of errors involved and to also discuss the idea of a system generally and how to improve it.There is, in the American system of justice, a presumption of innocence. That is, the burden of proof that a crime was committed rests with the government (the prosecutor). In a research situation, the burden of proof rests with the advocate of the research hypothesis (e.g. a researcher may assert that this drug will work to cure this disease)Using our criminal justice example, a first hypothesis (in statistics usually called a null (no difference) hypothesis) would state that the person who has been apprehended is innocent. This hypothesis can only be rejected if sufficient evidence of guilt is produced. In the research example, the null hypothesis (as it is called) would be that the drug does not work to cure the disease.The second (alternate) hypothesis is that the individual is guilty. The government takes this position, that’s why they arrested the individual. The question is can the prosecutor produce enough evidence of guilt to establish guilt. The burden, as we have said, rests with the prosecutor. In the drug example, the second hypothesis is sometimes called the research hypothesis and in our drug case, it would be that the drug does cure the disease.There are two mistakes we can make. We can let a guilty person go free. Or, alternatively we can call a person guilty who is, in fact, innocent. Or, we can say the drug works when it doesn’t, or we can fail to detect that the drug does work and conclude that it doesn’t.Usually in statistics those are called Type I and Type II errors. Or, ‘Errors of the First Kind and Errors of the Second Kind. To be clear, Errors of the First Kind are to mistakenly reject the no difference (null) hypothesis. Errors of the Second Kind would be to failing to reject the no difference hypothesis when there was actually a difference (the person is guilty)So, in the criminal justice case, we have a trial. In the research case, we do a research study.The innocence or guilt of the person at trial is not known. We do not know for certain if the drug works. That is the key. If we knew for certain, we wouldn’t need a trial.And because of that uncertainty we will make those mistakes.Statistics has been called a tool for “…making decisions in the light of uncertainty…” If we had a sure fire way to know the innocence or guilt of the individual, we wouldn’t need the trial or study. But we are uncertain.So we set a standard of proof. A pre-selected point or criterion that, when met, will be sufficient to say we will make a decide this way or that. In the U. S. criminal justice system, that standard is ‘…beyond a reasonable doubt’. In other words, the null hypothesis (the presumption of innocence) has to be rejected beyond a reasonable doubt.It is important to realize that this standard is arbitrary. There is nothing about it derived from theory that makes one standard more valid than another. Obviously the selection of the standard will affect the frequency with which one makes the two mistakes.It is popular to try to avoid making the two mistakes. Cries of outrage are heard whenever a person thought to be guilty is freed and we find (particularly since the advent of DNA testing) that non-guilty people are sometimes convicted in spite of their evident innocence.It may be no consolation to the victims in these cases, but from a system point of view it is important to understand that these mistakes are inevitable. They are a function of the uncertainty. Only in the most extreme circumstances can either of these mistakes be eliminated and that is by committing the other mistake as often as possible.If society never wants to convict an innocent person, don’t convict anyone. But the maximum number of guilty people will go un-convicted. If society never wants to let a guilty person go free, then convict everyone. But the maximum number of innocent people will be convicted.Outside of those extreme cases, the mistakes are unavoidable. They each will be committed; one once in a while and the other once in a while.Thus the aim of any study framework should be to try to achieve a balance.One more note before moving on. If we fail to reject the no difference hypothesis, it does not necessarily follow that the alternate hypothesis is true. Just because we don’t convict the individual in court does not mean that he or she is innocent. It means that we could not meet the burden of proof. Because the standards are arbitrary they can be set in such a way as to make meeting the burden of proof easier or more difficult.Part II – How it works in statistics

Six Sigma III

John — Sun, 29 Jun 2008 20:21:11 +0000

Imagine you are an executive in an organization that defines quality as absence of defects. How, then, would you plan to improve quality?It’s obvious isn’t it? Spend time being clear about what are defects and what are not (via specifications), find points to check for defects and, if they are found, remove them. Institute a corrective action process attempting to avoid recurrence of the defect.Now, put a different hat on your head. You are an executive in an organization that defines quality as continuous improvement. What do you do to assure quality?Define quality in terms of the customer (the only eyes that matter) would be the first step. Second would be to analyze processes in the organization to see to what extent they achieve the quality definition. Then, begin a continuing effort to reduce variation and center processes on target.Note that the approaches are fundamentally different in emphasis. Naturally, activities overlap and both kinds efforts involve teamwork, training, coordination and leadership. But the focus is different.Now, reconsider the competitive track record of organizations that define quality as a never ending process of continuing improvement and the competitive track record of those companies that chase defects.There really is no comparison. Those companies adopting the continuous improvement paradigm (e.g. Toyota, Honda, Canon) have maintained high levels of quality, lower or comparable costs and better values over the years and their profitability shows it.The Six Sigma approach is chasing defects. Finding and correcting problems is not improving quality, it is merely getting process to operate the way they were originally intended to run. Zero defects is not good enough in today’s competitive world. Not when you are competing with someone that does not stop trying to improve once the defects are gone.To repeat: It’s all fairly simple really. Pay attention to quality in the right way and quality improves while costs go down. Enter the market with better quality at lower cost and capture the market.There is no longer the slightest doubt that this method is the key to competitive excellence.

Six Sigma Part II

John — Tue, 24 Jun 2008 06:28:22 +0000

As we saw on Part I, there are some fundamentally flawed conceptual problems with Six Sigma and its statistical underpinnings are shaky indeed.But the aspect of Six Sigma that makes it that makes it least useful as a tool to improve competitiveness is that it is based on defect detection and elimination (reduction) and not improvement.Removing the defects from a process does not improve it. It merely allows it to operate the way it was intended to operate. It is somewhat counterintuitive, but one cannot improve a process by reacting to its outcome. That is what Deming is saying in the 3rd of his 14 points, “Cease dependence on inspection as a means to improve quality.”Of course removing defects is desirable. And if there are a lot of defects, rework, scrap, etc., eliminating it will be all to the good, but that doesn’t change the fundamental operation of the process. In order to do that, one must concentrate efforts on working upstream.Over the years in consulting I have always differentiated between two types of variables. Those that are process outcomes; what the customer sees. I call these Key Quality Characteristics. Upstream are the variables that, in some combined and interactive way produce those results. I call these Fundamental Process Variables.For example, a key quality characteristic of a magnetic disk for a disk drive might be it’s flatness. In today’s demanding environment, recording disks must be extremely flat. The customer demands it. But disk flatness is a result of a machining process. In that sense it is an outcome. To make a disk more flat it is necessary to understand the variables of the upstream process that govern flatness; the grinding pressure, the abrasive particle size of the grinding slurry, etc.Detecting disks that aren’t flat (defects) does not make disks flatter. To improve disk flatness requires action on the fundamental process variables.This seemingly semantic difference may seem trivial, but on the contrary, it drives improvement efforts and ultimately determines the level of competitiveness a company can attain. Many companies execute well, but there’s more to competitiveness than that.Again we can look to Deming for the fundamental message. One of my favorite quotes of his is, “It’s all very simple, really, pay attention to quality in the right way, costs will go down and you enter the market with the best quality at the lowest cost. You will capture the market.” This is from an interview he gave to the New York Times many years ago. It is also from the message he brought to Japan in 1950. It is as true today as it was then, the difference being that today there is no doubt that it is true.Next is Part III - How the approach to improvement determines competitiveness.

Six Sigma: Some problems - Part 1

John — Sun, 22 Jun 2008 05:01:01 +0000

One can hardly walk into a bookstore these days without being deluged with Six Sigma volumes of one sort or the other. It seems to be an answer for every businessman’s problem from controlling quality to designing new products to guaranteeing gleeful customers. As is usually the case, with these fads, the truth is somewhat less than the claims. Will adopting Six Sigma make a manufacturing company more effective? Is this a direction your organization should take? What about “Lean Six Sigma”? Is that the wave of the future? Six Sigma attempts a measure of process capability. Indeed the estimation of six sigma itself is based on this concept. But, a process that is drifting into and out of control does not have a predictable capability associated with it. Six Sigma statistical calculations do not require a statistically stable process. There is no requirement to assure a stable process average or process variation and this results in estimates of six sigma that vary significantly depending on when the sample used to make the calculation was drawn.Six sigma advocates indicate that detecting a 1.5 sigma shift is an adequate safeguard for this problem but as Donald Wheeler (the world’s foremost expert on statistical process control) shows in his paper, “The Six Sigma Zone”, no such assurance exists. In fact, this causes inappropriate action, searching for trends when there are none and ghost hunts. Finally the calculation of six-sigma itself is accomplished by dividing a denominator based on a subjective assumption (The number of opportunities over which a defect can occur) into a measure of the number of defects where defects have been so ill-defined as to produce no meaningful measurement. Also the so-called area of opportunity is essentially infinite. In a an unstable process, problems can arise from many locations. As far as I can tell, operational definitions are not used or advocated in any literature that I could find in my thorough search. Yet, as Deming points out, they are vital.For example perhaps there is a requirement in a restaurant that the customers’ tables be wiped clean before they are seated. Therefore a ‘dirty’ table is a defect. But, what does this word ‘dirty’ mean. No casual food lying on it? No standing water? Clean enough to eat off of without a plate? Clean enough for surgery? What does it mean to say the table must be clean? It means nothing until you say what it means operationally – in this case, for this purpose. This operational definition is a must. The statistical conversion from a defect rate (assuming a meaningful one can be found) to a probability density function that can yield percentage estimates of areas under a curve is too tortuous to discuss here. Suffice it to say that this too, has serious statistical faults.Finally there is no mention in the Six Sigma literature of some important scientific principles. The elements of prediction are not discussed. Operational definitions (which are critical to training and measurement) are not discussed anywhere that was apparent. Logical thinking is not mentioned. The dangers of copying and other Post Hoc fallacies (e.g. confusing correlation with causation) are not discussed. Hypothesis testing is taught as a statistical method with no mention of the serious shortcomings associated with hypothesis testing and prediction. In short, there is not the emphasis on scientific and statistical thinking that will be an integral part of the new strategy.

Wikipedia

John — Thu, 19 Jun 2008 22:29:37 +0000

An interesting (if somewhat lengthy) piece on Wikipedia from the New York Review of books…here.

(more…)

E Factors and Control Charts

John — Tue, 17 Jun 2008 04:49:58 +0000

Briefly, there are many ways to estimate the variation of a sample of values. One is to pull out your calculator and follow the instructions for calculating the standard deviation. Typically that is what they call σ (the instructions usually have that wrong. n-1 is used for s not σ ). That estimate is of all the variation and includes both common cause variation and special cause variation if it is present. It is not the estimate we want to use to set control limits for a control chart used to detect special causes.

(more…)