Evaluating the Arguments of Experts FAQ 1.0

Often in the debates and discussions which occur in the religion oriented news groups, various experts are cited in support of one proposition or another. The natural question is then "how do I know which expert is better?" This FAQ is meant to help the nonspecialist evaluate the arguments and testimonies of "expert witnesses." Although it is specifically directed toward the religion oriented news groups, the principles are useful in evaluating the remarks of an expert in almost any field.

This FAQ is "in progressu" and all comments and criticisms are appreciated.

The following comments, originally posted by Kevin W. Davidson, sparked my interesting in composing this FAQ, and will be used as an outline for the FAQ.

Now how can someone decide the truth when two experts are arguing?

One approach would be to become an expert oneself. But that might take a while.

Another approach is to compare credentials of the experts, although different people might evaluate those credentials differently

Finally one might just count the number of experts (or expert books) on each side of the issue. This has many problems including classifying who is an expert, and also the problem of obsolete information.

One might try weighing the evidence presented by the two experts, but even this may be difficult if one or both of the experts is misrepresenting the primary sources he uses in his argument.

What I personally look for is one of the authors using persuasion techniques not based on logical argument, but on the skills of the propagandist.

If none of these work, the argument will remain unresolved in my mind.

I don't have a simple solution, only a warning. The facts aren't always what they appear to be, and evaluating expert opinion isn't as easy as it might seem either.

Kevin (kwdavids@mercury.interpath.com) | http://www.interpath.com/~kwdavids/

1. Should I try to become an expert myself?

1.1 One way, as Mr. Davidson suggests, is to become an expert oneself. In most cases this is impractical - the time and study it takes truly to become an expert in a given field of study these days is quite extensive, and this is no less true in areas related to theology and biblical studies as it is in any other area. However, if one is really interested, one can take the time at least to obtain an introduction in order to understand the basic questions, definitions, and parameters of research which are involved. One should remember that even introductory texts to a subject are written from a particular point of view, but it is often instructive to see the author answering objections which differ from his own perspective.

2. How do I choose the best introductions to a subject?

2.1 Try posting such a question to one of the news groups in which the subject of interest is often discussed. There are also the library and the various search tools available on the Internet.

2.2 Examine the book. Look through the table of contents and read the preface and introduction. Scan the first paragraphs of several of the chapters. Does the book seem to address the questions and issues in a clear and factual manner? Does the writer acknowledge and address diverse points of view?

2.3 It never hurts to read more than one book on the subject.

3. Often somebody's credentials are listed - so what?

3.1 Credentials are usually listed to establish the credibility of the person advancing the argument. They are not irrelevant. For example, who is more likely to be right when discussing Shakespeare - the one who has a B.A. and M.A. in English, and has written a thesis on Shakespeare, or the one who has majored in Biology, and has managed to read most of Shakespeare's plays (and not all of them), and none of the Sonnets?

3.2 The credentials should be relevant to the subject being discussed, or at least related. A Ph.D. in Physics does not qualify one to expound authoritatively on philosophy. Often, a person may have studied a subject as an ancillary discipline to his or her main field, so that the physicist has studied a fair amount of math and science, or the church historian has had a better than average introduction to Old Testament studies.

3.3 The entire credentialing process, especially in the United States, assumes that the individual who possesses the credentials has been evaluated by those who are competent in his field. Has the individual obtained his degrees from accredited institutions which have high academic standards? Or has he gotten them from a degree mill or little known institutions? This does not automatically disqualify a person: one can buy a good product from a bad store, but it should give us pause as we consider the individual's claims.

3.4 Nevertheless, credentials are often the most trumpeted when one has the weakest arguments. This is particularly the case, it seems, in news groups.

4. What if 2000 experts agree with one side, and only 2 agree with the other?

4.1 The problem is that the two might be right. It has happened before in the history of scholarship!

4.2 However, it is unlikely. The usual assumption is that the minority position is the minority position for a reason. One way is to evaluate how the other experts react to the minority position over a period of time. If the minority position begins to gain ground, it may be that a "paradigm shift" is in the making.

5. How can I weigh the evidence presented by experts?

5.1 This can be difficult if one lacks a knowledge base in the subject (see point 1). One suggestion is to take the time to verify the facts the experts are using in support of their arguments. One can try to find a "neutral" third party who discusses the issues and utilizes the same facts.

6. How do I evaluate the actual arguments which the experts are using?

6.1 I am glad you asked. In a sense, all the comments listed above are preliminary. It is possible even for a non-expert to spot logical fallacies and bad arguments, if one knows what to look for. The following are drawn principally, but not exclusively, from 2 sources.

John Frame, *The Doctrine of the Knowledge of God*, (Phillipsburg, NJ: Presbyterian and Reformed, 1987), p. 242-301.

D.A. Carson. *Exegetical Fallacies*, (Grand Rapids: Baker, 1984), p. 91-126.

The definitions are reworded and the examples are usually my own, and I have provided several not addressed in either source. Another helpful resource is the alt.atheism FAQ "Constructing Logical Arguments," though it is very incomplete. I have restricted this FAQ to a discussion of the fallacies involved in arguments. If one is interested in a broader discussion of the proper use of logic in arguments, then Frame has a good introduction.

6.2 Argument by assertion. This involves simply making a statement and providing no supporting evidence. The most frequent fallacy in news group discussions! This is somewhat natural - often people in such discussion are simply spouting their opinions. However, when an expert is arguing his or her case, one expects more. If called upon, the expert should be able to support the conclusion with evidence and valid arguments. Failure to do so may indicate that the individual does not have support for the position.

6.3. Threat of force. This sounds bad, but the usual form is "if you do not accept this argument, something bad may happen to you." For example, someone may argue "If you do not accept the King James Bible as the only valid translation, you will not be accepted in this church." The fallacy here is fairly obvious - that one will experience unpleasant consequences for rejecting an argument does not mean that the argument is true.

6.4. Ad hominem. A very popular debate tactic, the ad hominem seeks to debunk the argument by attacking the person who holds the argument. There are subcategories:

6.4.1 Comparative or abusive. This attacks the argument by attacking the people who have held the argument. For example "You should not believe in Christianity, because some Christians have killed people who disagree with Christianity." The problem with this is that it doesn't prove that Christianity is true or not: it simply proves that some Christians have behaved badly. This is similar to the false dichotomy argument, or the excluded middle - what caused some groups to kill others for disagreeing with them might be their *misunderstanding* of Christianity.

6.4.2. Circumstantial ad hominem. This assumes that an argument should be believed or disbelieved because of special circumstances.

Because you are a Harvard student, you should study more.

Since your denomination supports abortion, how can you criticize mine for supporting abortion?

6.4.3 Character assassination. This fallacy seeks to undermine an argument based on some alleged flaw in the individual's life, or that because the individual has flawed arguments elsewhere, his or her arguments here are bad.

Because Bruce Metzger does not hold to the inerrancy of the Bible, all of his scholarship is false and unreliable.

There were communists and liberals involved in the translation of the RSV. Therefore, the translation cannot be trusted.

6.5 Argument from silence. This argument claims that something is true because it has not been proven false, or vice versa.

The New Testament does not address the issue of abortion. Therefore, abortion is permitted.

This fallacy often operates at the semantic level, i.e., because something is not mentioned by name therefore nothing contributes to the discussion. In the case of the argument above, other considerations would come into play, such as the New Testament's positive support of the Old Testament definition of "personhood" and "life."

6.6 Appeal to emotion. This tends to evoke emotional responses of the reader or hearer to the proposition, usually by using "loaded" terms, or by making associations and connections which are either positive or negative. It usually takes the form of presenting one's arguments in the best possible light, and one's opponents' in the worst.

The democratic platform is dynamic and progressive, just the thing to lead us triumphantly into the 21st century!

Fundamentalism is in reality a species of neo-gnosticism which maintains rigid theological conclusions having no relevance to the real world.

Note how appeal to emotion can be used to bolster other weak arguments, such as argument by assertion, or ad hominem. Making people like or dislike a proposition does not prove the truth or falsity of the proposition.

6.7 Appeal to authority. See point 3 above. In reality, all of us depend on authorities all the time, once we are satisfied that the person really is an expert on the subject. False appeals to authority would include celebrity endorsements (just because Michael Jordan likes the shoe does not make it the best product). Authorities who contradict themselves, misuse evidence, or use fallacious arguments are also suspect. Often it takes another expert to point this out.

6.8 Causality. People often try to argue based on a misunderstanding of cause and effect (which David Hume argued could not really be proven to exist, but we'll ignore that for the moment). There are several types.

6.8.1 False cause, or *post hoc, propter hoc* (after this, because of this). Because two things are closely associated temporally does not mean that they are related. The relationship must be proven by showing further factual connections.

Every time you visit a city, there are car accidents there. You jinx!

When Clinton was elected, the crime rate dropped several percent.

6.8.2. Genetic fallacy. This assumes that the original state of something and the current state are necessarily the same. Again, this cannot simply be assumed, but must be proven with additional evidence, because development and change do occur.

Because the Greek word originally meant "immerse," the meaning must be the same in the New Testament (ignoring that words can sometimes change meaning over the course of time).

The human tendency to look around corners stems from the fact that we evolved from apelike creatures who were often hunted by large predators (another false disjunction as well: the behavior could be a conditioned response).

6.8.3 Aristotle distinguished the following types of causes, and one still sees these distinctions being made in theological and philosophical argumentation, though infrequently at the popular level.

Efficient (that which makes something happen - the popular definition).

Final (the purpose for which something happens).

Formal (most essential quality that makes something what it is).

Material (that out of which something is made).

Occasionally, one sees someone using the concept of "cause" ambiguously when a more precise definition would help.

6.8.4 Oversimplification of Cause. Often an effect can have multiple causes and contributing factors.

This country is in the fix it's in because the Supreme Court ____________ (fill in your favorite pet peeve).

Economics is the sole explanation for World War II.

6.9 Generalization. Similar to 6.8.4, one should always be wary of arguments which use the word "all" followed by some sort of predicate characterizing the "all."

All pit bulls are vicious.

Hitler was motivated by a fallacy: "All Jews are evil."

6.10 Complex question. This fallacy gives you two or more for the price of one - questions that is. It combines questions that really ought to be considered separately and may or may not be related.

Are you one of those unthinking atheists? (It should be 1) "Are you an atheist?" and 2) "Are atheists unthinking?").

6.11 Equivocation. This means that terms are not being used consistently throughout the argument.

Some dogs have fuzzy ears. My dog has fuzzy ears: therefore, my dog is some dog ("some" is being used in two different senses).

6.12 The part for the whole, or composition and division. This fallacy assumes that what is true of a group is true of an individual, or vice versa.

Mark comes from a church which teaches that being a democrat is the unforgivable sin. Therefore, Mark believes the same thing.

Because Christ commanded the church to evangelize the whole world, therefore, I must evangelize the whole world.

6.13 Denying the antecedent and affirming the consequent. Sorry about the "formal" logic terms, but most people can intuitively spot these:

Presbyterians believe in predestination. John is not a presbyterian, therefore he does not believe in predestination (even if the premise is true, the conclusion may be false).

If the power goes out, the computers will go out. The computers are off, therefore, the power is out.

In more popular terms, this could be called "negative or positive inference." In other words, we cannot assume from a positive proposition that the negative is true, nor from the negative proposition that the positive is true. It must be proven, since there are usually external qualifications which may prove the assertions false or true.

6.14 False disjunction, or misuse of the excluded middle. The excluded middle simply means that a proposition is either true or not true. The problem arises when there is a third or multiple alternatives to the assertion. A famous one cited in the alt.atheism.faq is C.S. Lewis's argument in *Mere Christianity* (I paraphrase)

Jesus was either God, or a liar, or a lunatic.

A third possibility is that the gospels give us a completely false, revisionist perspective on Jesus, so that we don't even know whether or not anything we read about him is true. Another example:

You either support the program of our pastor, or you are not a Christian.

The individual who fails to support his pastor may have cogent reasons for not doing so which have nothing to do with his spiritual state.

6.15 Failure to make or observe valid distinctions. This fallacy assumes that because two things are alike in some aspects, they are therefore alike in all aspects.

Because children and adults have equal rights before the law, they therefore should be allowed to vote and drink.

This ignores that children do not have the intellectual and emotional maturity to make informed decisions, or the physical capacity to handle alcohol in even moderate amounts.

6.16 Appeal to selective evidence. The individual only cites evidence which supports his own viewpoint, and ignores evidence which does not. A competent, honest scholar will seek to interact with views which are contradictory to his own position, limited only by space and the overall purpose of his work.

6.17 Invalid syllogisms. Often we intuitively know that an argument is wrong. Reducing the argument to a syllogism will sometimes help. It may be that the premise of the argument is not truly universal in fact. The famous:

All swans are white.

Jack is a swan.

Therefore Jack is white,

is an obvious example (although valid in form, all one has to do to prove the argument false is to produce a black swan). An example that Carson (p. 100) offers:

No false teaching possesses inherent authority for the church.

Some teaching is false teaching.

Therefore, no teaching possesses "inherent" authority for the church.

The minor term is distributed in the conclusion, but not the premises. There is a big jump from "some" to the implied "all."

6.16. The world view, or "my frame of reference the only frame of reference" fallacy. This interprets facts and material arising from one world view in terms of another world view.

The shaman claimed that he believed in the biblical resurrection from the dead, but he meant that all people eventually come to life as spirit-beings . . .

which is quite different from the biblical view of a bodily resurrection. However, it can get very subtle. Good authors can so write as to get you to at least conditionally accept their presuppositions, so that in the frame of reference in which they are writing, their conclusions and interpretations make sense, but still may not be appropriate to the original context. This fallacy is frequent in literary and biblical interpretation.

6.17 Confusion of truth and precision. This fallacy arises from our actual use of language. The statement "I am a man" and "I am a male homo sapiens 37 years of age" are both true, but one is far more precise than another. This is the fallacy behind the alt.atheism.faq's argument concerning the precise dimension of the Bronze Sea in 1 Kings 7:23.

6.17 The non sequitur. This is a conclusion which is not supported by or related to the argument and evidence which is provided. Some are easy to spot. Others are based on false premises or confused thinking that are not easy to untangle.

Paul teaches that there is no distinction between male and female, and therefore would permit women to teach in the church.

The argument does not follow because it does not specify what Paul means by "no distinction between male and female," and this must be addressed before the conclusion is drawn.

6.18 Circular Argument. Many will point out that there is a sense in which all thought is circular, i.e., we all argue from certain presuppositions which cannot be ultimately proven to everyone's satisfaction. However, in most of our arguments, some principle (evidence) external to the argument has to be applied to validate the argument.

The existence of chaos indicates that God doesn't exist.

Why do you say this?

Because chaos is proof that God doesn't exist.

7. Concluding remarks.

The purpose of this FAQ, as stated above, is to aid the reader in evaluating the arguments and evidences provided by experts in support of their contentions. Future editions of the FAQ (there will be at least one more iteration) will include a bibliography and a brief definition of logic and a discussion of its proper use in argumentation.

I would also like to make several observations that I couldn't fit in above. Much of the above simply formalizes what is intuitive and obvious to anyone who has a modicum of education. In evaluating the arguments of experts, one should also consider the tone and style that the writer is using. Is it excessively rhetorical? Is there a real nastiness or hostility? Such stylistic marks may reveal an unhealthy bias on the part of the writer which harms rather than helps his case. As Mr. Davidson puts it, does the writer use the techniques of propaganda, or does he provide evidence combined with valid arguments in support of his position, interacting fairly and soberly with diverse points of view, including alternative interpretations of the evidence? Does he quote his opponents fairly and not out of context so that he subtly changes their arguments? One should also beware of a writer who depends heavily on debunking the arguments of others without concomitantly proving his own.

You may either post your criticisms and comments or address them to:

nebarry@netaxs.com

I am especially interested in any major fallacies which I may have left out, and any possible improvements to what I have written.

Hi Barry,

One big fallacy which you didn't directly address is that of the a priori fallacy, "trying to base knowledge of fundamental synthetic truths on anything other than empirical evidence", as my encyclopedia puts it.

Errors in the use of statistics and probability are common, though not so much on the religion boards.

Arguing from analogy is always a formal fallacy if it is used as a deductive argument.

And the "fallacy of the saving hypothesis", explatory excerpt:

From a secton subtitled "Fallacies in nondeductive reasoning and in observation" from the article "Fallacies" from _The Encyclopedia of Phiolosopy_ as instructive:

We may add a fallacy of saving hypotheses. It is certainly a fault for a thinker to be so attached to a hypothesis that he notices only evidence that agrees with it and ignores or denies unfavorable evidence. Popular superstitions of all kinds are protected by this fallacy, but it is also common among scientists, historians, and philosophers. It may also be a mistake, when one finds evidence that is prima facie unfavorable, to introduce supplementary ad hoc hypotheses in order to protect the original one from falsification. Carried to an extreme, this procedure constitutes a linguistic change that makes the original hypothesis analytically true, and it can generate the fallacy described above of oscillating between an analytic and a synthetic use of the same expression. In less extreme cases, how can we systematically mark off this error from the respectable procedure of interpreting new observations in the light of an extablished theory? Perhaps in two ways: first, in the respectable procedure, we are working with a hypothesis that is already well confirmed, but it is a fallacy to "save" a hypothesis for which there is no strong independent support; and second, even if the original hypothesis confirmed, it may be appropriate to consider, after it has been "saved" by additional hypotheses (after the new observations have been interpreted in the light of the original hypothesis) or has been modified and qualified in various ways, whether some alternative hypothesis would account better for the whole body of evidence.

I found your FAQ clear and, I should think useful.

If it helps a few people think more clearly, then it will have been worth the effort.

Ciao.

Michael Fisher

In a previous article, nebarry@netaxs.com (Barry Hofstetter) says:

[ all of your text deleted to save room ]

Barry: Thank you for taking the time and trouble to post your FAQ. To me they are a reminder of more active discussion times...

May I share 2 items which have served me well over the years to remind me of much of what you posted?

1. [a folk story, of sorts] President Lincoln one day was in a heated discussion with some Senators, when he paused to ask his hearers: "Gentlemen. Do you see that cow outside the window?" They looked out of the Oval office and said they did. "Now, for the sake of discussion, let's call the tail of that cow a leg. Now then, how many legs does the cow have?" "Why, Mr. President, that's simple. Five legs, obviously." "Ah, gentlemen," Lincoln pointed out, "that's where you and I differ; calling a tail a leg doesn't make it one."

2. [an anecdote from my personal experience] During my student days in France, two of my classmates, one French, the other Swiss, got into a hot argument over some issue unimportant here. After a few minutes of high-volume exchanges, one of my French friends interrupted just as loudly: "Allez-y! Allez! Quand on n'a plus de raisons, on s'affirme!" Loosely translated: "Way to go! When you run out of reasons, you assert yourself!"

Hope you enjoy the stories --- and good luck on your candidature

(and orals?) ---

Camille

(A copy of this message has also been posted to the following newsgroups: soc.religion.christian.bible-study)

In article <4r3l76$2hm@netsrv2.spss.com>, nebarry@netaxs.com (Barry Hofstetter) wrote:

[...]

| I am especially interested in any major fallacies which I may have left out,
| and any possible improvements to what I have written.

I recommend the book "Don't you believe it" by, I believe, Hoover. Unfortunately, I've misplaced my copy and can't check the author's name, although the title is correct. It's a small work, in paper, which goes through many, many examples of logical fallacies.

__|_______                  Through His love and through a ram,
  |                           He saved the son of Abraham...
  | Bob Felts               But God demonstrates His own love for us,
  | wrf3@mindspring.com       in that while we were still sinners,
  | http://www.mindspring.com/~wrf3            Christ died for us.

A word of general caution, however, about the study of informal fallacies. Absent a solid understanding of logical consequence or validity (and a healthy dose of discernment), a round of Spot the Fallacy can quickly take on the sad, if somewhat comic, features of a snipe hunt. Fallacies can suddenly appear around every corner, even where they ain't. And teaching informal logic can prove dangerous -- as Max Schulman's Dobie Gillis learned when his sweet nothings were thrown back by his prospective significant other as just so many reasoning miscues.

Trouble is, these fallacies aren't always fallacious. Elsewhere I distinguished between a valid argument and a valid _form_ of argument, where an argument form is valid just in case it has (or can have) none but valid arguments as instances. (You all know, like the backs of your hands, that a valid _argument_ is one such that it's impossible that it have true premises and a false conclusion.) This distinction is important here. At 6.12 Barry treats the fallacies of Composition and Division, which have, respectively, the following forms:

Each part (or member) of X is F; hence, X is F [Composition]

X is F; hence, each part (or member) of X is F [Division].

(Presumably 'part' here is to be taken in the sense of _proper part_, so that the whole isn't a part of itself. Otherwise Composition can never have any invalid instances.) These forms are clearly fallacious: they each have instances that lead from true premises to a false conclusion. (Consider sets. Each member of the set of persons subscribed to REFORMED is a person. That set, however, is no person. Of course that set _is_ an abstract object -- a property which is not shared, presumably, by _any_, let alone each, of its members.) But there are arguments of these forms that are perfectly valid. E.g., consider that object o'er yonder:

Each of its parts is made of solid gold (weighs more than a ton, is located in Helena, Montana); whence, it is made of solid gold (weighs more than a ton, is located in Helena, Montana).

It is impossible that the premise of this Composition argument be true and its conclusion false. Likewise the following Division argument:

California is located in North America; whence, Long Beach, California is located in North America.

So there are indeed valid Composition and Division arguments; those forms don't invariably give rise to fallacious arguments. But the same goes for _argumentum ad hominem_ or appeals to pity or any number of other forms of argument that are standardly labelled as fallacies in informal logic texts. Accordingly one should strive to resist being a (knee-) jerk in applying them. There are, to be sure, purely mechanical procedures in logic. But they tend to be the exception, not the rule, and in any case are conspicuous in their absence in the matters of argument identification and classification.

In sum, then, two things. Learn logic and its rules well; but rule those rules with wisdom and discernment. And second, commend Barry for his work here (as elsewhere). Barry is also to be commended for the uncommon subtlety of his humor:

pmartin@csulb.edu
/////////////////////////////////////////////////////////////
Philosophy, California State University, Long Beach, CA 90840