Six Sigma: Some problems – Part 1
Posted Under: Deming,General Management,Statistical Thinking
One can hardly walk into a bookstore these days without being deluged with Six Sigma volumes of one sort or the other. It seems to be an answer for every businessman’s problem from controlling quality to designing new products to guaranteeing gleeful customers. As is usually the case, with these fads, the truth is somewhat less than the claims. Will adopting Six Sigma make a manufacturing company more effective? Is this a direction your organization should take? What about “Lean Six Sigma”? Is that the wave of the future?
Six Sigma attempts a measure of process capability. Indeed the estimation of six sigma itself is based on this concept. But, a process that is drifting into and out of control does not have a predictable capability associated with it. Six Sigma statistical calculations do not require a statistically stable process. There is no requirement to assure a stable process average or process variation and this results in estimates of six sigma that vary significantly depending on when the sample used to make the calculation was drawn.Six sigma advocates indicate that detecting a 1.5 sigma shift is an adequate safeguard for this problem but as Donald Wheeler (the world’s foremost expert on statistical process control) shows in his paper, “The Six Sigma Zone”, no such assurance exists.
In fact, this causes inappropriate action, searching for trends when there are none and ghost hunts. Finally the calculation of six-sigma itself is accomplished by dividing a denominator based on a subjective assumption (The number of opportunities over which a defect can occur) into a measure of the number of defects where defects have been so ill-defined as to produce no meaningful measurement. Also the so-called area of opportunity is essentially infinite. In a an unstable process, problems can arise from many locations.
As far as I can tell, operational definitions are not used or advocated in any literature that I could find in my thorough search. Yet, as Deming points out, they are vital.For example perhaps there is a requirement in a restaurant that the customers’ tables be wiped clean before they are seated. Therefore a ‘dirty’ table is a defect. But, what does this word ‘dirty’ mean. No casual food lying on it? No standing water? Clean enough to eat off of without a plate? Clean enough for surgery? What does it mean to say the table must be clean?
It means nothing until you say what it means operationally – in this case, for this purpose. This operational definition is a must. The statistical conversion from a defect rate (assuming a meaningful one can be found) to a probability density function that can yield percentage estimates of areas under a curve is too tortuous to discuss here. Suffice it to say that this too, has serious statistical faults.Finally there is no mention in the Six Sigma literature of some important scientific principles. The elements of prediction are not discussed.
Operational definitions (which are critical to training and measurement) are not discussed anywhere that was apparent. Logical thinking is not mentioned. The dangers of copying and other Post Hoc fallacies (e.g. confusing correlation with causation) are not discussed. Hypothesis testing is taught as a statistical method with no mention of the serious shortcomings associated with hypothesis testing and prediction. In short, there is not the emphasis on scientific and statistical thinking that will be an integral part of the new strategy.
Reader Comments
As both a physicist and a Six Sigma Black Belt, I find this a very interesting, yet deeply flawed, post.
For instance, the calculation of process capability (Cpk and related measures) requires, by definition, a stable process. Similarly, while one definition of “defects per million opportunities” requires subjective measures, Six Sigma practitioners are taught that variable data is much preferable to attribute data, which means “defect” and “opportunity” are generally defined by clear specifications. Even where such specifications may be arbitrary, they are not subjective, and the improvement process forces the organization to reconsider them. Where ambiguity exists, as in your example, the organization is driven towards operational definitions in order to resolve the very questions you raise. In other words, any definition of a defect is better than no definition, and the application of Six Sigma forces organizations to drive out variation in the definitions.
The shortcomings of practitioners’ depth of statistical and scientific understanding is a real problem, as you indicate, but what is the alternative? Six Sigma is implemented in organizations where such knowledge does not exist and is neither understood nor valued by senior management. The alternative to Six Sigma is not better scientific and statistical thinking; the alternative is continued non-scientific management. It is unlikely that most organizations would be willing to expend the resources needed to develop full scientific and statistical skills in their employees (e.g. by sending employees to attain at least a four-year degree), or that most employees would be willing to undertake such an effort. Six Sigma improves scientific and statistical thinking in people with a wide range of backgrounds and keeps the drain on an organization’s resources at an acceptable level. While not perfect, it is a huge step in the right direction, and opens the doors for organizations to implement scientific thinking throughout its operations.
Hi tom,
Thanks for your comment. I’ll respond shortly. Please stay tuned for Part II. Some of the valid issues you raise will be addressed.
I would make a couple of quasi-technical points. As a general statement, the statistical methods taught by Six Sigma books and consultants are, for the most part, in error.
Yes, one should have a stable process to use a cpk as a prediction. But you and I both know that few of the users of cpk index values check process stability or report on it. Without that information, the number is meaningless.
But that is not what I was really getting at. Inherent in the very calculation of Six Sigma is a (mistaken) conversion of defect rate to a process capability model. Not only is this bad practice statistically, in the Six Sigma literature that I’ve seen (admittedly I haven’t read it all) there is no reference to the problem of citing a defect rate from a non stable source…no distinction is made when calculating that defect rate. Then that rate is used to form (mistakenly) a process capability to a given process.
The same is true with regard to the definition of area of opportunity and defect. With regard to defect, an operational definition is required. There is no other way to achieve a communicable meaning to a term like round, safe, clean, in-spec, etc. The same requirement exists for designating, in a communicable way, an area of opportunity for a defect to occur.
The most popular Six Sigma website (http://www.isixsigma.com/) barely mentions this vital need and the definition they give of operational definition is not correct. Another site (http://www.6sigma.us/) when searched returns no finding for operational definition. A third site, Motorola’s, does not have operational definition in its “Six Sigma Dictionary”.
Sigma is given as calculated using the total sample sum of squares. This is not correct unless the process is free of special causes and without a control chart one has no knowledge about the presence or absence of special causes.
Hypothesis testing is advocated (e. g. t Test for independent samples). First there is no reference that I could find that such a test has no meaning for prediction unless the samples are from a process in control. Moreover the pitfalls associated with hypothesis testing (type I and type II error for example) are virtually never explained.
It seems to me that the non-technical portion of your remarks falls into two categories. First you say that the “…improvement effort forces…” and later “…Six Sigma forces organizations…” I disagree. What forces the organization to do things is management. Many companies that do not ascribe to Six Sigma pay close attention to process stability, consistency of definitions, etc.
Second, you argue that there are no alternatives to some of the bad practices of Six Sigma. I disagree with that as well. There are very good alternatives. Toyota and Canon do not use a ‘Six Sigma” approach to quality or competitiveness and each company has been consistently been among the very best at quality for the products they produce and they have both been consistently very profitable in the same world markets that the U. S. has written off.
It has been said that, “The proof of the pudding is in the eating.” Using that homely advice we can look at Motorola and GE as Six Sigma exemplars and Toyota and Honda as non-users of Six Sigma and see that there is no comparison. General Electric is trying to sell its small appliance division. What a sad piece of news that is.
Tom’s comment that ‘ “defect” and “opportunity” are generally defined by clear specifications ‘, highlights Six Sigma’s most fundamental flaw. Defects and specifications say nothing about the process. Indeed, they can produce any values one wishes. It is hence a philosophy based on wishful thinking !
3.4 dpmo is just too laughable for anyone with an once of common sense to take seriously. It was first based on a “drift” then a “correction” then a “dynamic mean off-set”. Anyone intested in the farce behind its derivation and the various attempts to prop it up, is invited to read my papers :
http://qualitydigest.com/IQedit/QDarticle_text.lasso?articleid=12564
http://qualitydigest.com/IQedit/QDarticle_text.lasso?articleid=12541
The extent of Six Sigma’s flaws are too numerous to describe here. My papers give greater detail.
Tom goes on to naively ask ‘what is the alternative?’ … has Deming been forgotten so soon ?
John, you raise some good points. I agree that the websites are generally atrocious, and I have rarely used the calculation of Sigma, except for occasional reporting purposes. Perhaps I have been lucky enough to have been trained by an unusually good MBB.
I will just add a clarification. As you point out, there are excellent alternative improvement methodologies. I meant that there are few practical alternatives for introducing and building statistical and scientific thinking across an organization. I believe that this is a valuable goal, beyond Six Sigma’s efficacy as an improvement methodology. In my experience, Six Sigma offers training and implementation of statistical and scientific thinking that most people, from senior managers to line employees across the organization, can comfortably swallow.
Hi Tom:
Yes, I’d say you have a better idea of the problems with some uses of sigma and cpk than most. In my experience, showing process stability before reporting cpk (Six Sigma efforts or not) is the exception and not the rule.
Also I think you are right to point out that it is easy to let the perfect be the enemy of the good. Clearly if some organization is paying no attention to quality and not quantifying data and they adopt a Six Sigma mehtodology (or almost anything like it) they’ll see improvments and be better off.
It is just frustrating to see a very effective and time-proven set of techniques like those Toyota has been using since Taiichi Ohno worked there, be ignored for an ineffective and theoretically flawed method.
Especially now that we have some years to look at what Six Sigma has done for some its strongest proponents, we can see it has not come even close to helping those companies and Toyota, Honda, etc. just keep rolling over everything in sight.
Hi All,
I am new to Six sigma.
Am currently working in a software project which requires six sigma concepts.
For determining quality of a software product
I have used DMPO .
Is there any other concepts for determining the
quality of a software product.
Can i use any statistical tool ?
Please advice..
Thanks
Lavanya
I suggest you start by making a list of questions you’d like to answer. The statistical tools that are most appropriate will depend on the nature of the questions asked.
Hi John,
Thanks for your reply
I need six sigma for finding defects in a web application and to find the solution on how to reduce the defects.
I have two main questions require answers.
Question 1
I have used DPMO for
calculation the defects in given opportunity
For Root Cause analysis i made
use of Pareto Analysis.
These two were not sufficient for web applications.
Is there any other methods i can use for
1)Calculating defects
2)Root Cause Analyzing tool
Question 2
1)Can i use sampling methods(Random,Convenient,Purposive)
If so which is best suited.
2)For analyzing data can i use any tools(Chi-square,Percentage Analysis)
It will be really helpful if u could answer the above questions.
Even if your answers give me an idea of these,it will be useful.
Thanks in Advance
Lavanya
Hi,
What is the question?
Is this question why you have defects? Is the question how many are there? Why do you want to know that?
You have, perhaps inadvertantly, found one of the basic weakenesses of the six-sigma approach. What a defect is and what the area of opportunity is are both arbitrarily defined and one can make a report look better or worse by changing the definition.
But that is just a game.
I assume you wish to eliminate/reduce errors.
Where do they come from? Is there a stable pattern of errors (statistical control chart). If there is not a stable pattern when are the peak occurrences? If there is a stable pattern, then something systemic must be changed.
You list a lot of tools, but no questions. It is hard to know which tool makes the most sense if one doesn’t know what question he or she is trying to answer.
Of greater concern to me is when a company rushes into 6 sigma without addressing for instance people factors or without reviewing built in assumptions of systems in place. Shortly before he died Dr. Deming himself said no one understood his QM philosophy. On many occasions he says you can’t pick only a few points to apply as his 14 points are very interwoven much like an ecosystem.
So today glaring issues emerge. Few seem to be addressing Deming’s 7 deadly diseases. Stability of systems should be first goal but isn’t. Leaders of some companies talk nonstop on costs/profits/productivity while riding a 6 sigma campaign saying slogans like ‘no errors, no errors’ to workers. That’s a shame. There does seem to be less of a real look at what systems as a whole today are doing. Instead of focusing on systems the focus is on individuals creating defects while speaking more than ever before. I’m concerned that more people are getting stuck to PC/technology solutions gathering even more data while doing less small group meetings than ever before. This loss of real interaction could lead to more worker conflicts while more company leaders pride in being a ‘technology company’ thereby escaping their real issues.
Finally I do think more emphasis in Quality these days should done be on defining/refining the desired targets (real variables instead of looking for more errors) more clearly so that fewer ‘errors’ can naturally emerge.