Environmental decision-making and the precautionary principle:
what does this principle mean in environmental sampling practice?
A.J. Underwood
Landscape and Urban Planning, Vol. 37, pp. 137-146 (1997)
Precautionary principles require that environmental managers should be careful about making decisions where there is ignorance about the underlying issues. If a mistake is going to be made, it should be "in favour" of long-term environmental welfare. For example, don't let a runway be built if there is uncertainty about environmental impacts. Such principles require particular care in technical aspects of sampling to detect environmental impacts or in studies on which to base predictions about future impacts.
Statistical analyses of variable ecological data particularly require attention to two types of mistake. The first (or Type I) is concluding there is in an impact when, in fact, there isn't one. The second (or Type II) is the conclusion that there is no impact when there is one. Either can happen because samples in the affected habitat are not perfect measures of what is really happening. So, precautionary principles require that Type II errors should be prevented, because it is not cautious to miss real impacts. This paper describes how sampling should be designed so that the information obtained is more precautionary and less likely to cause mistakes that do environmental damage.
TOP OF PAGE
Environmental decision-making and the precautionary principle:
what does this principle mean in environmental sampling practice?
A.J. Underwood
Landscape and Urban Planning, Vol. 37, pp. 137-146 (1997)
Increasingly, environmental decision-making is scrutinised with respect to a precautionary principle. This principle asserts that where uncertainty and doubt make it impossible to be sure about a correct decision, any errors should favour the long-term sustainability of the environment.
Although there are problems in practical adherence to this principle, it has particular meaning and value for quantitative, ecological aspects of environmental sampling and monitoring. When probabilistic (or statistical) interpretations of data are made, there are two potential errors: Type I, or rejecting a null hypothesis when it is true and Type II, or retaining a null hypothesis when it is wrong. In environmental terms, these can often be translated as Type I error occurring when it is claimed that there is an environmental impact when there is none. Type II error would represent failing to detect an impact even though one has occurred.
Most ecological and environmental work is designed to keep the chance of Type I error small (and by convention at about one in twenty). Usually, there is little to no concern about Type II error. The precautionary principle, however, dictates that Type II errors are a serious problem for environmental management - and much more so than Type I error. Thus, not detecting impacts (Type II) is not precautionary.
This paper summarizes the relevant features of environmental monitoring and sampling that decrease the chance of Type II error (and therefore increase the "power" of a sampling program to detect impacts). Better use of these issues in the design of sampling would greatly increase adherence to the precautionary principle and would enhance the prospects of sustainable environmental decision-making.
Interpreting precautionary principles in terms of environmental sampling and measurement would increase the need (and potentially the capacity) to define possible environmental disturbances and responses to them in more quantitative and less vague terms.
This document was generated using the LaTeX2HTML translator Version 96.1-h (September 30, 1996) Copyright © 1993, 1994, 1995, 1996, Nikos Drakos, Computer Based Learning Unit, University of Leeds.
The command line arguments were:
latex2html Week11.tex.
The translation was initiated by Yung-Pin Chen on Tue Dec 8 08:37:34 EST 1998
Yung-Pin Chen
Our Conversation Weeks 10 and 11 (due: Tuesday, Dec. 1)
Question 1. Please explain type I and type II errors.
Question 2. In criminal cases, the usual legal standard is ``proof beyond a reasonable doubt.'' In civil cases where monetary damages are at stake, the general legal standard is a ``preponderance of evidence.'' When judges in the Eastern District of New York were asked to assign probabilities that correspond to proof beyond a reasonable doubt, their answers ranged from 76% to 95%; for a preponderance of evidence, seven judges answered `+50%'' with the others responding 50.1%, 51%, and 51%. Use the concepts of type I and type II errors to explain these different legal standards.
A2) In our society, being held for a criminal offence not commited is a harsh mistake on the part of the judge, jury, and lawyers. In a criminal case, the prosecution must prove beyond a reasonable doubt that the person being held and tried for the indescression is the person who committed the crime. If the officials trying the case put an innocent person behind bars, that is a type I error. These errors are seen as very serious, therefor the judges require a larger amount of evidence to convict a person of the crime. In civil cases there is less at stake. Money is not as big an issue as a human life. By rejecting a true null hypothesis, as presented by the prosicution, the judge would be committing type II error, which has lesser consequences, therefor has a smaller amount of evidence required to convict.
2) Type I error could have been made when judges were assigned to measure the probabilities that correspond to the proof beyond a reasonable doubt. Since each of the judges have different perspectives on the proof for validity, the answers could range from 76% to 95%. Type II error could have been made when judges were assigned to measure the probabilities based on the evidence of ponderance, because each of them could have failed to notice that the evidence was actually false.
A2: Proof beyond a reasonable doubt is important because a defendent could be convicted of a crime that s/he did not commit. It order to prevent this mistake, the proof should be very strong, hence the 76%-95%. If the null hypothesis is a "not guilty" verdict, and it is rejected in favor of a "guilty" verdict, then Type I error has been made. However, the phrase "preponderance of evidence" suggests that only enough proof is needed to indicate a 50% chance of guilt. The word "preponderance" implies that the evidence does not have to be as inarguable as the instance of needing proof beyond a reasonble doubt. This is where Type II error could be committed; Type II error, it would seem, is not as serious as Type I error. It is easier with verdicts to committ Type II error than to committ Type I error. This is okay: the American judicial system will assume one's innocence until guilt is proven.
Question 2: Since the consequences of criminal cases are more serious than those in civil cases, it is more important to be certain that the accused people are actually guilty if convicted. Therefore, if we assume that the null hypothesis is that someone is guilty, to convict someone in a criminal case, we want to have a small type II error. It is for this reason that the legal standard for criminal cases is "proof beyond a reasonable doubt." As the judges in th Eastern District of New York interpruted this, to convict someone of a criminal offense, they must be at least between 76% and 95% certain (the exact percentage varied between judges) that he or she is guilty. This would mean that there is no more than a 5% to 24% chance of type II error. On the other hand, this creates the possibility of having a large type I error since the accused person will be set free if there is even a doubt that they are guilty. In a civil case, however, the situation is different. Because the punishment for being convicted is only a fine, not as severe a punishment as in a criminal case, the consequence of making a type II error is not as great. Therefore, simply having a "preponderance of evidence" is sufficient enough to convict someone. According to a sample of judges from the Eastern District of New York, this corresponds with a probability ranging from "+50%" to 51%. While this leaves a rather large chance of type II error, the probability of type I error is reduced. Because the person will be convicted if there is at least a 50% to 51% certainty that he or she is guilty, there is a lower probability of making a type I error because the defendant will only be not guilty (and H1 accepted) if there is less than a 50% to 51% certainty.
Answer to Question 2: In criminal cases in which "proof beyond a reasonable doubt" is needed to convict a defendant,the assigned probability of 76% to 95% makes it very difficult for a type II error and more probable to make a type I error. The question being, of course: is the defendant guilty? In civil cases in which a "preponderance of evidence" is needed, the assigned probability of "+50%" makes it easier to make a type II error and harder to make a type I error. The thinking here is probably that it would be a far greater crime to rob someone of their freedom rather than rob them of their money.
-Type II Error is defined as accepting Ho when Ho is false.
Because there needs to be a greater probability to have proof beyond a reasonable doubt i.e. 76%-95% it will be less likely to have either Type I or Type II errors since one must be more certain that one's proof is validated.
There will more likely be type I and type II occuring concerning a preponderance of evidence as the probability needs only to be +50%.
Within the legal system, type I & II errors can explain mistakes than occur. The null hypothesis of the American legal system is "innocent until proven guilty." This hypothesis, while rather conservative, is good to have because it can be argued that it is better to let a guilty man go free than to imprison an innocent man. A type I error in this case would occur when a man was convicted a crime he did not committ (very serious mistake). A type II error would be when he was found not guilty and actually was. The judges questioned all indicated that over 50% of the time evidence will prove guilt beyond a reasonable doubt. The one judge who said 95% of the time evidence will prove guilt beyond a reasonable doubt is very sure of this! Nevertheless, they all indicate, even with a number such as 50.1%, that the majority of the time the evidence will be enough to prove guilt beyond a reasonable doubt.
2. Ho= doubt(not proven guilty) Hi=no doubt(guilty) In the case of a criminal trial, if the null hypothesis is true(there is doubt that someone is guilty), then rejecting it means an innocent person goes to jail. If the null hypothesis is not true but is not rejected, then a guilty person goes free. Ho=not enough evidence Hi=proponderance of evidence In a civil trial, a type one error occurs when a person is charges with a crime even though the evidence didn't suggest they committed it. A type two error occurs when a subject is released or set free of charges when the evidence strongly suggests they committed a crime.
TYPE II ERROR: is when you accept the null hypoth. when it is false. For ex., w/ the coin toss, the null is that someone flipped, you accept this null as true, but the person really faked it. Here, the alternative hypoth. is true & you made the wrong decision.
CRIMINAL CASES:
If we let the null hypoth = not guilty while alternative hypoth = guilty,
In terms of Type I & II errors: If one rejects the null (not guilty) & you're wrong, it's a TYPE I-more serious cause sent an innocent man to jail/execution. On the other hand if you accept the null (not guilty) when s/he's guilty (make wrong decision) you put a criminal on the street-Type II error. Since the judges say "proof beyond a reasonable doubt" = Pr = 76-95%, it follows that the chance of a TYPE I error is small (which is good since we hope to not send innocent people to jail) because the probability is high of proving the case beyond a reasonable doubt.
A chart would look like:
| accept null | reject null | |
| null (not guilty) | OK | TYPE I |
| alternative (guilty) | TYPE II | OK |
CIVIL CASES: With the civil cases the null and alternative hypoth's are the same-definitions of errors same. But this time the Pr of being sure of the decision is only approx. 50%-much lower than the above's 76-95%. Hence, the chance of a TYPE I ERROR (here, for ex., making an innocent person dish out alot of ) in civil cases is much larger. But then again the stakes seem lower w/ civil cases (vs. imprisonment/death/mean inmates!)
If the null hypothesis is not correct and we accepet it, that is called a type II error and it is not very serious kind of a mistake.
Question 2. In the criminal cases the judges have to be more confident about their decision. Because if they make a wrong decision they might send an innocent person to the jail. In this case if the null hypothesis is correct and they reject it, that would be the type I error, which is really serious. So, they have to be more confident and when they are asked to get probabilities that would correspond to proff beyond a reasonable doubt their answer varied from 76% to 95%.
On the other hand in civil cases when the judges do not have to send people to jail, it is okay for them to be less confident. So, in this case the null hypothesis they set can be wrong, and if they accept it, that would not be a serious mistake.
Question 2: If the null hypothesis is that the defendent is not guilty, and in actuality, the defendent is not guilty, but the defendent ended up in jail, then this is a Type I error. On the other hand, if the null hypothesis was that the defendent is guilty, yet the jurors or the judge failed to reject the null hypothesis, and let the defendent free, than it is a Type II error.
A2: Criminal vs. civil cases: "Beyond a reasonable doubt" could be said to involve a higher p value than "preponderance of evidence." The p value in criminal cases needs to be higher than that in civil cases in order to be significant. "Reasonable doubt" appears to be quantified as at least 76% certainty, whereas "preponderance" is at least 50%.
A2. When the judges were asked to assign probabilities to proof beyond a reasonable doubt, they in a sense created a null hypothesis. They said that anything above about 76% was proof enough to constitute innocence. Anything below this percentage would be the alternate hypothesis. This can lead to both type I and type II error in that some cases may have been tampered with thus sending an innocent person to jail, or setting a criminal free. The same thing is true when the judges were asked to assing probabilities to the preponderance of evidence. They all basically responded with a percentage at or above 50%. Thus, they decided that roughly half the time their null hypothesis, being that there is not enough evidence, is correct and it gets rejected resulting in type I error. The other half of the time their null hypothesis gets accepted when in fact it is false and thus type II error results. Basically the result of this study shows that the legal standards are not concrete, and that it really depends on the type of case that is being examined. In the civil cases there seems to be more of a chance that the wrong decision is made than in the criminal cases.
Question 2: When trying to prove something "beyond a reasonable doubt" it is much easier to make a type II error than a type I error. Because type II errors are less severe than type I errors the probabilities are more confidently predicted as higher. To prove something on "a preponderance of evidence" is much more difficult; therefore the probabilities are much lower because there is less confidence that a type I error will not occur.
When someone is on trial a type II error is considered the most serious error. That is, if the charge against someone is accepted as true, and it turns out later that they are actually innocent, this is a serious mistake of the legal system. People are supposed to be innocent unless proven guilty. Therefore, the probability that a certain person is guilty had better be much greater than half, and therefore the most likely outcome, if a person is to be convicted. Hence the large probabilities. When dealing with money, rather than human lives, such errors are less serious. A type I error or a type II error are about equally as serious. Either way, someone is being deprived of money, rather than deprived of years of their life. Therefore, it is only necessary for the probability of the hypothesis being correct to be more than half.
Question 2. In criminal cases, the stakes are very high; a group of twelve people make a decision about a stranger's life. That decision could cost the person several years of his life, or, in many states, it could cost him his life. The probability associated with "proof beyond a reasonable doubt" needs to be very high to correspond to the seriousness of a type I error taking place. A high probability means there is a greater chance of a type II error happening, but this is viewed as less serious a mistake than a type I error in a criminal case. In a civil case, the stakes are different. Although whichever way the jury goes will affect the people involved, it will cost them money and not their lives. In a civil case, therefore, there needs to be enough evidence, but not as much as in a criminal case. This means that the chance of either a type I or type II error taking place is higher than in a criminal case. The reasons for the difference in the two cases range from the difference in cost to the people involved to the amount of civil cases that are tried in courts every year.
Question 2. A type 1 error is much more serious than a type 2 error in both of these situations because if the judge assumes that someone is guilty but they are actually innocent than that person will be wrongfully punished. A type 2 error is somewhat less disastrous because if that person is dumb enough to commit a second crime chances are that they will be caught. The different legal standards can be explained because being convicted of a criminal crime is generally more detrimental to one's reputation than a civil crime. Also the legally enforced punishments for criminal cases are much more severe than those imposed in civil cases.
Question 2: Perhaps the probability for "proof beyond a reasonable doubt" is higher because these are usually more serious cases where a person risks going to jail. If the wrong verdict is made, then there is a worse outcome. The "preponderance of evidence" standard probably has a lower probability because there is less at stake and evidence may be easier to obtain.
The difference in legal standards is probably related to the severity of the case. In a criminal case the sentence being discussed is usually jail time, community service and even death, thus the probabilities would be quite high to achieve Rproof beyond a reasonable doubt.S In civil cases the sentence is usually an exchange of money so the probabilities are lower to achieve a Rpreponderance of evidence.S Type I and type II errors can relate to this because these errors can sentence a person to death or set a criminal free or, in a civil case, punish or fail to punish an innocent or guilty person.
An instance where the null hypothesis is false (the diference between the two populations does not equal zero), but we answered yes (concluding that the difference between the two vales equals zero) we have created a type II error.
The judges in the eastern district of New York assigned a probibility corresponding to "proof beyond a reasonable doubt" of 76%-95%. This means that there is a 76%-95% chance that the jury will commit neither a type I or type II error and the verdict, either guilty or not guilty, will be just. However, there is a liklihood that the jury will find the defendant guilty when he is really innocent (commiting a type II error) or that the jury finds a guilty man innocent (commiting a type I error). The fact that all of the judges, with a variance of only 1%, agree on the percentage for preponderance of evidence means that they all agree it has a 50/50 chance of occuring. That there is a 50% chance you will commit a type I error and a 50% chance that you will commit a type II error.
The differnce in the probalities that correspond to proof beyond a reasonable doubt lies in the difference between criminal and civil cases. Criminal cases result in more serious consequences, due to crimes of more serious natures. The proof beyond a reasonable doubt needs to be convincing. If we look at the situation from the perspective of type I/II error we see the difference. If a person we charged as guilty but was in fact not guilty, the consequence would be virtually irrevocable. Civil cases regarding less serious crimes need proof sufficient to tip the scales to one side or the other. The consequences of such a crime make type I error still a serious error, but less of one than in a criminal case.
The two legal standards, "proof beyond a reasonable doubt" and "preponderance of evidence" are statements in response to the null hypothesis. The criminal statement declaring "proof beyond a reasonable doubt" has a corresponding probability of 76%-95%. This, assuming the null hypothesis is true, is an example of a Type I error. They are saying that the defendent is guilty, when he is indeed innocent. They are making this statement with a lot of confidence which is reflected in the high probability percentage. The civil statement that declares a "preponderance of evidence" claims the probability percentages of 50%, 50.1%, 51%, and 51%. This is an example of a Type II error. They are saying that the defendent is innocent when he is actually guilty. They are less confident about their statement which is refelcted in the probability percentage that could go either way; they admit a greater possibility of being wrong. In real life, it would be a very serious mistake to convict someone who is actually innocent, and in addition be proven wrong after making the decision with such confidence. The greater the confidence the more tragic the error. The percentages in both cases describe the probability of observing data outside the null hypothesis. They give statistics to the "proof beyond" and that certain "preponderace". A "preponderance" indicates there is enough evidence to make a statement, but "proof beyond" indicates there is enough evidence to make a statement with confidence. Since a null hypothesis is in itself a conservative conjecture, to make a mistake against the truth of it, or a Type I error, is to offend the possibility of revealing the truth.
A Common Misconception |
In a statistical test, the researcher selects between two mutually exclusive hypotheses: the null and the alternate hypothesis. It is a common notion that:
- You don't believe in the null hypothesis
- You do believe in the alternate hypothesis
In this article I argue this logic and explain why it is incorrect.
The Logic of Falsification |
The notion of disbelieving in the null hypothesis is based on the principle of falsification introduced by prominent philosopher of science, Karl Popper. According to Popper (1959), we cannot conclusively affirm a hypothesis, but we can conclusively negate it. The validity of knowledge is tied to the probability of falsification. For example, a very broad and general statement such as "Humans should respect and love each other" can never be wrong and thus does not bring us any insightful knowledge. The more specific a statement is, the higher possibility that the statement can be negated. For Popper, a scientific method is "proposing bold hypotheses, and exposing them to the severest criticism, in order to detect where we have erred." (Popper, 1974, p.68) If the hypothesis can stand "the trial of fire," then we can confirm its validity.
Quantification such as the assertion that "the mean of population A is the same as the population B" is a high degree of specification. Following the Popperian logic, the mission of a researcher is to falsify a specific statement rather than to prove that it is right. Therefore, the attempt of falsification leads to the disbelief of the null hypothesis.
Why null? |
Careful readers may ask, "Why do we distrust and try to falsify the null hypothesis only? Why don't we apply the same action to the alternate hypothesis?" Indeed, current hypothesis testing procedure is a hybrid of schools of Fisher and Neyman/Pearson. Testing the null hypothesis was introduced by R. A. Fisher (1949) while the alternate hypothesis was suggested by Neyman and Pearson (1928).
We can specify the null hypothesis easily, but we don't know what exactly the alternate hypothesis is. We may hypothesize that there is a mean difference between the two populations, but we cannot point out how wide the gap would be. We don't even know from which of the alternate population the test statistic comes from. At most we can say that the difference is not zero.
Indeed, the logic of hypothesis testing is: Given the null hypothesis is true, how likely it is for the ocurrence shown by the data to surface? When the p value is 0.0001, it means that 1 out of 10000 times the data will surface as it did under the assumption of the null.
Because we are confined to start with the null hypothesis only, hypothesis testing is not a fair application of Popperian logic of falsification.
Reality |
In reality, we can always find problems with the notion of disbelieving in the null hypothesis. Stevens (1992) gave a good example: Suppose a medical researcher conducts a study to examine the safety of a new drug. His hypotheses would be:
- Null: The new drug is unsafe
- Alternate: The new drug is safe
In this case the doctor should tend to doubt with the alternate hypothesis rather than the null, because if the researcher mistakenly rejects the null and the drug is indeed unsafe, this mistake would cost human lives! In other words, it is a fatal Type I error. There is a real life example in Europe: Once the tranquilizer thalidomide was claimed to be safe but actually the drug was dangerous to pregnant women (cited in Miller & Knapp, 1978).
Balancing Type I and Type II errors |
It is an ongoing debate about the proper use of hypothesis testing. When we use hypothesis testing, we should be aware of the weakness of the logic. Blindly disbelieving the null hypothesis is unwise. Instead, a careful researcher should balance the Type I and Type II error. Neyman and Pearson (1933a), who introduced the concepts of Type I and Type II errors, receommended that controlling Type II error should be favored in scientific research. Ludbrook and Dudley (1998) argued that in biomedical research it is advisable to control Type I error.
There isn't a clear-cut way for balancing these two errors. The following story illustrates how subjective values would affect the weighing of the hypotheses:
Once a warship is patrolling along the coast. Suddenly an unidentified aircraft appears on the radar screen but the computer system is unable to tell whether it is a friend or a foe.
The captain says:
- The null hypothesis is that the incoming aircraft is not hostile. If it is indeed hostile and I don't fire the missile, it is a Type II error. The consequence of committing this Type II error is that we may be attacked and even killed by the jet.
- The alternate hypothesis is that the incoming aircraft is hostile. But if it is not hostile and I shoot it down, it is a Type I error. The consequence of making this Type I error is the termination of my career in the Navy.
- It seems that the consequence of Type II error is more serious. Therefore, I disbelieve in the null hypothesis. Fire!
The commander shouts "Delay the order!" He argues:
- If the null hypothesis is false but we don't react, the consequence is that a few of us, let's say 30, may be killed.
- If the alternate hypothesis is false and actually the incoming aircraft is a commercial airliner carrying hundreds of civilian passengers, the consequence of committing a Type I error is killing hundreds of innocent people and even starting a war that may eventually cause more deaths.
- I assert that the consequence of Type I error is more severe. Thus, I disbelieve in the alternate hypothesis. Hold the fire!
The above story is exaggerated to make this point: Subjective values affect balancing of Type I and Type II error and our beliefs on null and alternate hypotheses. The founders of hypothesis testing, Neyman and Pearson (1933b) asserted that there is no general rule for balancing errors; in any given case, the determination of "how the balance [between Type I and Type II errors] should be struck, must be left to the investigator." On the contrary, Lipsey (1990) gave a specific guideline: In basic research it is desirable to keep the probability of Type I error low. It is because the nature of basic research is that the researcher should be very conservative about accepting new facts or changing facts of existing knowledge. On the other hand, in applied research it is preferable to minimize the Type II error rate because in a situation where effective treatment is needed and not readily available, a Type II error can represent a great practical loss.
References |
Fisher, R. A. (1949). The design of experiments. London: Oliver and Boyd.
Lipsey, M. W. (1990). Design sensitivity: Statistical power for experimental research. Newbury Park: Sage Publication.
Ludbrook, J. & Dudley, H. (1998). Why permutation tests are superior to t and F tests in biomedical research. American Statistician, 52, 127-133.
Miller, J. K. & Knapp, T. R. (1978). The importance of statistical power in educational research. (ERIC Document Reproduction Service No. : ED 152 838).
Neyman, J. & Pearson, E. S. (1928). On the use and interpretation of certain test criteria for purposes of statistical inference. Part I and II. Biometrika, 20, 174-240, 263-294.
Neyman, J. & Pearson, E. S. (1933a). The testing of statistical hypotheses in relation to probabilities a priori. Proceedings of Cambridge Philosophical Society, 20, 492-510.
Neyman, J. & Pearson, E. S. (1933b). On the problem of the most efficient tests of statistical hypotheses. Philosophical Transactions of Royal Society;, Series A, 231, 289-337.
Popper, K. R. (1959). Logic of scientific discovery. London : Hutchinson.
Popper, K. R. (1974). Replies to my critics. In P. A. Schilpp (Eds.), The philosophy of karl Popper (pp.963-1197). La Salle: Open Court.
Stevens, J. (1992). Applied multivariate statistics for the social sciences. Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers.