Entries Tagged 'Research Methods' ↓

To continue, we need to define a couple of terms. The first is a probability density function and the second is a sampling distribution.A probability density function expresses a particular function in terms of integrals. Thus for a frequency distribution smoothed (over repeated sampling) to form a curve as shown below, the area under the curve can be calculated and the probability of a given value occurring in the distribution can be assessed as a proportion of the amount of the curve that is to the left and/or to the right of the value. In a normal distribution, z values are used to do this.

Continue reading →

A hypothesis is a supposition made as a basis for research or reasoning without regard for its truth. So says the Oxford dictionary. What starts the hypothesis testing process is just such a supposition. In the justice system example we made, there is a supposition on the part of law enforcement that the apprehended person committed the crime.In pharmaceutical testing, there may be a supposition a given medicine will reduce cholesterol values in blood tests.Because of the nature of induction we can never prove theory going forward. As a prediction we recognize that the samples of interest are not those of the past, but rather those of the future and they have not happened yet. Thus they are not available to be sampled for our study and they may, in some way, be different from the samples that are available for the study.Further, that difference could render the results that are (necessarily) based on the samples in the study inapplicable to the future samples. This problem is unavoidable. So we design a way to test our hypothesis in another way. One aspect will be that the statements we can make will be probabilistic. We are in the land of uncertainty.To do our test we create another hypothesis called the null hypothesis. The Null hypothesis is essentially an assumption that the research supposition (Research Hypothesis) has no merit. In our justice system case, it is an assumption of innocence. In the pharmaceutical case the null hypothesis would state that the new drug had no effect on lowering cholesterol.So there are two hypotheses. The null hypothesis (usually designated H0 and the research hypothesis which is usually, and somewhat unfortunately, called the alternate hypothesis (usually designated H1 or HA). I say unfortunately because this is the hypothesis that we are most interested in and the word ‘alternate’ makes it seem almost secondary.Thus the comparison is constructed in such a way as to pose the null hypothesis and the burden of rejecting that (no difference) hypothesis rests with the advocate of the alternate hypothesis. That is, if the research can provide enough evidence to reject the null hypothesis, there is reason to believe that the new drug does have an affect on cholesterol. If the prosecutor can provide enough evidence we will suggest that the apprehended person did commit the crime.We could be wrong. People sometimes ask a statistician to give a yes or no answer or to provide certain proof. That simply cannot be done and that is a problem fundamental to the inductive nature of this process. It is for this reason that we do not accept the alternate hypothesis in the sense that it is ‘proven’. The logic of hypothesis testing this way leads to either reject the null (no difference) hypothesis or to not reject it.A standard of proof is given. In the justice example in the United States that standard is that the proof must be ‘beyond a reasonable doubt’. In a research study, such as the pharmaceutical example, we set probability level (more about this later).If that standard is met or exceeded we reject the null hypothesis. If the standard is not met we do not reject it.

When discussing hypothesis testing, I have, from time to time, used the example of the criminal justice system to act as a metaphor for the logic and philosophical issues involved. It remains a good way to pose the logical dilemmas and the types of errors involved and to also discuss the idea of a system generally and how to improve it.There is, in the American system of justice, a presumption of innocence. That is, the burden of proof that a crime was committed rests with the government (the prosecutor). In a research situation, the burden of proof rests with the advocate of the research hypothesis (e.g. a researcher may assert that this drug will work to cure this disease)Using our criminal justice example, a first hypothesis (in statistics usually called a null (no difference) hypothesis) would state that the person who has been apprehended is innocent. This hypothesis can only be rejected if sufficient evidence of guilt is produced. In the research example, the null hypothesis (as it is called) would be that the drug does not work to cure the disease.The second (alternate) hypothesis is that the individual is guilty. The government takes this position, that’s why they arrested the individual. The question is can the prosecutor produce enough evidence of guilt to establish guilt. The burden, as we have said, rests with the prosecutor. In the drug example, the second hypothesis is sometimes called the research hypothesis and in our drug case, it would be that the drug does cure the disease.There are two mistakes we can make. We can let a guilty person go free. Or, alternatively we can call a person guilty who is, in fact, innocent. Or, we can say the drug works when it doesn’t, or we can fail to detect that the drug does work and conclude that it doesn’t.Usually in statistics those are called Type I and Type II errors. Or, ‘Errors of the First Kind and Errors of the Second Kind. To be clear, Errors of the First Kind are to mistakenly reject the no difference (null) hypothesis. Errors of the Second Kind would be to failing to reject the no difference hypothesis when there was actually a difference (the person is guilty)So, in the criminal justice case, we have a trial. In the research case, we do a research study.The innocence or guilt of the person at trial is not known. We do not know for certain if the drug works. That is the key. If we knew for certain, we wouldn’t need a trial.And because of that uncertainty we will make those mistakes.Statistics has been called a tool for “…making decisions in the light of uncertainty…” If we had a sure fire way to know the innocence or guilt of the individual, we wouldn’t need the trial or study. But we are uncertain.So we set a standard of proof. A pre-selected point or criterion that, when met, will be sufficient to say we will make a decide this way or that. In the U. S. criminal justice system, that standard is ‘…beyond a reasonable doubt’. In other words, the null hypothesis (the presumption of innocence) has to be rejected beyond a reasonable doubt.It is important to realize that this standard is arbitrary. There is nothing about it derived from theory that makes one standard more valid than another. Obviously the selection of the standard will affect the frequency with which one makes the two mistakes.It is popular to try to avoid making the two mistakes. Cries of outrage are heard whenever a person thought to be guilty is freed and we find (particularly since the advent of DNA testing) that non-guilty people are sometimes convicted in spite of their evident innocence.It may be no consolation to the victims in these cases, but from a system point of view it is important to understand that these mistakes are inevitable. They are a function of the uncertainty. Only in the most extreme circumstances can either of these mistakes be eliminated and that is by committing the other mistake as often as possible.If society never wants to convict an innocent person, don’t convict anyone. But the maximum number of guilty people will go un-convicted. If society never wants to let a guilty person go free, then convict everyone. But the maximum number of innocent people will be convicted.Outside of those extreme cases, the mistakes are unavoidable. They each will be committed; one once in a while and the other once in a while.Thus the aim of any study framework should be to try to achieve a balance.One more note before moving on. If we fail to reject the no difference hypothesis, it does not necessarily follow that the alternate hypothesis is true. Just because we don’t convict the individual in court does not mean that he or she is innocent. It means that we could not meet the burden of proof. Because the standards are arbitrary they can be set in such a way as to make meeting the burden of proof easier or more difficult.Part II – How it works in statistics

Briefly, there are many ways to estimate the variation of a sample of values. One is to pull out your calculator and follow the instructions for calculating the standard deviation.    Typically that is what they  call σ (the instructions usually have that wrong. n-1 is used for s not σ  ). That estimate is of all the variation and includes both common cause variation and special cause variation if it is present. It is not the estimate we want to use to set control limits for a control chart used to detect special causes.

Continue reading →

I recently was in contact with a site that specializes in baseball statistics (Bill James’s Organization) and the fellow with whom I was in contact suggested a take a look at a unique graphics presentation.  I did look and was amazed with what I saw.  You can see it here.

Continue reading →