There is a maxim that the man with two watches never knows what time it is. The funny thing about this is that the man with one watch certainly doesn’t know any better than the man with two, he just thinks he knows what he does not. The man with two watches can maintain no such illusion of certainty because he has two watches with two (possibly ever so slightly) different times. He has been forced to recognize the existence of error. Socrates would be proud.
Any decent scientific work will spend a fair bit of time devoted to thinking about where the error is in the empirical work and what can be done about it. Many kinds of error can actually be measured. Thus we get opinion polls with +/- 3 percentage points attached to them.** This number reflects a certain kind of error in polls and that error can be measured sufficiently well to give one the +/- 3 points calculation.
What is frightening is when one sets academics or anyone loose to start offering theories which cannot be pinned down by any data. These are people with one watch. They have a theory; there is either no attempt to confront the theory with data or there is no ability to do so. The proffered attempts to prove the theory are laughably bad in their inability to actually prove or disprove anything. So instead of recognizing that the theory is unsupported, the theory becomes the One True Way because it is unrefuted.
In Book of Mormon studies, one sees this with people betting their souls on the error-ridden claims stemming from archaeological digs or literary analyses. Or people being convinced that Marxism, Feminism, Post-Modernism (whatever that means), Neoclassical economic theory, evolutionary origins to life, or whatever are perfectly correct and so judging the gospel or the world through that lens. It is fine to use these theories to think about things. But one should always end the discussion with a disclaimer like one gets in the opinion polls, “what I believe is true with a 90% chance, +/-90%.” If one doesn’t actually know what the chance is that what you are saying is correct, watch out! You’ve only got one watch.
This post is mostly true. In fact, I’d say that I am 80% sure that it is at least 75% correct.
** Opinion poll disclaimers actually represent an interval that is right nineteen times out of twenty. Thus the real disclaimer should be: “There is a 95% chance that the true number is within 3 percentage points of the number reported here”.
Arguing that a theory is the One True Way because it is unrefuted is the fallacy of “negative proof”. So that person can’t reasonably have any certainty at all, no matter how much he or she believes it.
Frank: It seems that you are implicitly assuming that the only way of verifying the truth of a theory is via empirical observation. It is far from obvious to me that this is true of all theories. Certainly, many ideas can be reduced to questions that are at least potentially empirically verifiable, but all of them?
Consider the statement:
“But one should always end the discussion with a disclaimer like one gets in the opinion polls”
Is the truth of falsity of this claim open to empirical verification? In theory, I suppose that the error rates of (some but not all) theories could be verified empirically. On the other hand, it seems that there is also some implicit premise here about the need for truth telling. Is that premise empirically verifiable? Perhaps the normative claim about truth telling can be reduced to an empirical claim of some sort, e.g. truth telling increases happiness. But then aren’t we suggesting that one should seek happiness? And how would one empirically verify this claim?
Obviously, normative claims are the low-lying fruit for this sort of counter argument, but I suspect that certain sorts of normative claims are nto the only claims that are not easily open to empirical verification. Consider, for example, the claim that in order to commence a civil rights action in federal district court one must exhaust one’s administrative remedies unless one is suing for racial discrimination under section 1981. Is the truth or falsity of this claim a matter of empirical observation? Not obviously so (but see Austin), nevertheless people every day come to conclusions about the truth or falsity of such claims. Furthermore, people seem to be able to discover that certain of these claims are clearly false — e.g. federal district courts have no subject matter jurisdiction over bankruptcy proceedings — and don’t seem to require strong qualification, despite the apparent absence of empirical verification.
Finally, those who engage in discussions aiming at discovering the truth of such apparently non-empirically verifiable statements do not seem to be engaged in either vacuous intellectual statements of the One True Way or unsuccessful attempts at empirical research.
Interesting comments, Frank. I’m trying to figure out what the alternative is to one betting her soul on empirical evidence. Do you consider prayer responses as empirical evidence subject to the same sort of possible error? Or do you see prayer response as some privileged type of intuition, free from the uncertainty you associate with the perception and measurement of real-world facts?
Interesting comments, Frank. I’m trying to figure out what the alternative is to one betting her soul on empirical evidence. Do you consider prayer responses as empirical evidence subject to the same sort of possible error? Or do you see prayer response as some privileged type of intuition, free from the uncertainty you associate with the perception and measurement of real-world facts?
Nate,
It is certainly the case that many interesting and possibly true things are not open to any form of verification. This does not make them untrue. It makes them untestable. Are they true? I don’t know. I addressed these claims in the sentence after the one you quoted:
“If one doesn’t actually know what the chance is that what you are saying is correct, watch out! You’ve only got one watch.”
And I think that is a valid concern. We should be humble about claiming certainty (even over obscure judicial claims). We should be hesitant to assume that such claims are more certain than claims based on faith. I am primarily concerned with those that wish to place man-made truth claims against gospel truth claims without thinking through the associated errors. We know our interactions with the Divine are plagued by our imperfection, yet sometimes forget that the same is true of our (and other’s) interactions with our environment.
This leads to Dave’s comment,
I think prayer is imperfect and empirical though not directly replicable. I think people receive false revelation on a regular basis. Perhaps some people think answers to prayer are always straightforward. I don’t know any of those people. Most of the time, people are a little bit amazed when prayer works immediately and directly. Thus everyone recognizes from their own experience that prayer has error. The problem comes when people don’t recognize that science has error too, and lots of it.
On another note, I think that faith is not simply a matter of empirical observation of the outcome of prayer. Faith can also be a granted feeling with no rational justification, given as a gift from God.
Lastly, I would add that I know of no scientific theory that has not required modification once confronted with the empirical facts. This is the history of science. It is a bizarre sort of blindness to think that our untested theories are true, when none of our tested theories ever, at first pass, have been.
Nate,
“Consider, for example, the claim that in order to commence a civil rights action in federal district court one must exhaust one’s administrative remedies unless one is suing for racial discrimination under section 1981. Is the truth or falsity of this claim a matter of empirical observation? ”
Suppose I look at how often this procedure was followed and how often it was not. This does not tell you what is normatively right but it, or a more careful analysis, could tell you whether or not circumventing this rule would get the case thrown out, which answers the relevant question as a probability. So abstracting from the normative, this seems verifiable in principle, even if perhaps not in practice.
Am I missing something?
“It is certainly the case that many interesting and possibly true things are not open to any form of verification. This does not make them untrue. It makes them untestable.”
My rather verbosely stated point is that I deny that empirical verification is the only kind of verification. I think that the truth or falsity of most important legal claims can be verified and tested. I don’t think that this is an emperical process. You seem to be assuming that verifiable must mean empirically verifiable.
Nate,
Do you mean you can show that the conclusions follow from the assumptions? What else do you have in mind as a test or verification?
Frank,
I’d like to point out that there is at least one way in which the truth (or falsehood) of a statement can be verified beyond a doubt without any experimental validation at all: it can be logically or mathematically proven or disproven. If you set up a system of axioms, then it is possible to make concrete, verifiable claims about the implications of those axioms without any data. Mathematics generally works that way.
Science sometimes fudges this issue as well; I can, for instance, plausibly claim that a certain object will accelerate at a given rate when a given load is applied to it without doing any experimentation to back up my claim. I get away with that because the underlying principles — in this case, either Newton’s Laws or the Law of Relativity, depending on the circumstances — are so well understood and so well proven that I do not need to prove them again in the process of making my claim.
Frank,
I’d like to point out that there is at least one way in which the truth (or falsehood) of a statement can be verified beyond a doubt without any experimental validation at all: it can be logically or mathematically proven or disproven. If you set up a system of axioms, then it is possible to make concrete, verifiable claims about the implications of those axioms without any data. Mathematics generally works that way.
Science sometimes fudges this issue as well; I can, for instance, plausibly claim that a certain object will accelerate at a given rate when a given load is applied to it without doing any experimentation to back up my claim. I get away with that because the underlying principles — in this case, either Newton’s Laws or the Law of Relativity, depending on the circumstances — are so well understood and so well proven that I do not need to prove them again in the process of making my claim.
Frank,
I’d like to point out that there is at least one way in which the truth (or falsehood) of a statement can be verified beyond a doubt without any experimental validation at all: it can be logically or mathematically proven or disproven. If you set up a system of axioms, then it is possible to make concrete, verifiable claims about the implications of those axioms without any data. Mathematics generally works that way.
Science sometimes fudges this issue as well; I can, for instance, plausibly claim that a certain object will accelerate at a given rate when a given load is applied to it without doing any experimentation to back up my claim. I get away with that because the underlying principles — in this case, either Newton’s Laws or the Law of Relativity, depending on the circumstances — are so well understood and so well proven that I do not need to prove them again in the process of making my claim.
Frank,
I’d like to point out that there is at least one way in which the truth of a statement can be verified beyond a doubt without any experimental validation at all: it can be logically or mathematically proven. If you set up a system of axioms, then it is possible to make concrete, verifiable claims about the implications of those axioms without any data. Mathematics generally works that way.
Science sometimes fudges this issue as well; I can, for instance, plausibly claim that a certain object will accelerate at a given rate when a given load is applied to it without doing any experimentation to back up my claim. I get away with that because the underlying principles — in this case, either Newton’s Laws or the Law of Relativity, depending on the circumstances — are so well understood and so well proven that I do not need to prove them again in the process of making my claim.
Oh dear, sorry for the multiple posts… my browser locked up in the middle of posting and I thought the post had failed! Won’t happen again.
Glen
Glen,
Thanks for the post. Mathematics can be used to show that A follows B, but never that B is true. Certainly math is still useful. It is as or more useful than empirical work. But once again, good mathematics is always very explicit about the assumptions made. Unfortunately, there is tendency for people to ignore the caveats and assert the conclusion without even knowing the asusmptions.
As for Newton’s Laws, these used to be considered universal, until they were modified by the Theory of Relativity. That theory is also constantly under revision. Understanding that this revision process occurs is exactly what I am getting at. We know how fast loads accelerate given a certain amount of force because we’ve tested it to death. 9.8 m/s^2 did not just pop into someone’s head as an untested theory.
Frank wrote: “Mathematics can be used to show that A follows B, but never that B is true.”
Where B is constrained to be a proposition of what form?
Where B is the axiom or set of axioms that remain unproven. Every proof has one. Most have lots of them.
So math reveals the conclusions embedded in one’s assumptions. But all mathematical conclusions are conditional on the truth of the assumptions.
Frank wrote: “Where B is the axiom or set of axioms that remain unproven.”
So if I insert this in your previous statement, it becomes “Mathematics can . . . never [be used to show] that unproven axioms are true.” I won’t argue with that.
“Every proof has one.”
Even if you’re proving a mathematical statement of the form “B implies A”?
“Even if you’re proving a mathematical statement of the form “B implies A”?”
Every proof is a conditional statement, starting from some set of assumptions or axioms (which I labeled B). Thus my point about being explicit about the assumptions.
I’m not sure what you are worried about, because my statement is pretty uncontroversial. Do you have something in mind?
This uncertainty principle also has applications in how we see ourselves and our relationships with each other. This is the gist of Michael Frayn’s play Copenhagen, which is a great application of the principle in human relationships. I wrote a little about it over at my new blog.
Frank wrote: “I’m not sure what you are worried about, because my statement is pretty uncontroversial.”
You said “Every proof has one [i.e., a set of unproven axioms].” I disagree, and so do you if you think “[m]athematics can be used to show that A follows B”. Suppose the mathematical statement P can be proved if one is allowed to assume the truthfulness of the set Q of unproven axioms. Then the mathematical statement Q->P (“Q implies P”) can be proved without assuming anything. (In particular, Q->P’s truthfulness doesn’t depend on Q being true.)
My favorite axiom is the theory of the second best, which is well accepted as proven.
Basically, it is the proof that progression from what is to the optimum is not linear.
For an example, if we know that no monopolies is our preferred state and we have seven, it is not a given that six monopolies or one monopoly is a better state that seven, even though that is the linear progression from seven to one.
Nate and Frank: Until Kant, it was generally believed that taste was not merely subjective, but certainly not empirically verifiable in any way that I think we would recognize. For example, counting up the number of people who like the Mona Lisa won’t tell you whether liking it is tasteful. Kant changed the notion of taste to something subjective and it has been ignored ever since–until H-G Gadamer.
Gadamer argues that the foundation for most of our knowledge is in something like taste. There is no public set of rules for deciding taste rationally, but taste is nevertheless a kind of knowledge and not merely a personal preference. He says, “taste knows something–though admittedly in a way that cannot be … reduced to rules and concepts” (Truth and Method 38).
The argument of the book is far too complex to be precised here, but it is relevant to your discussion. And, Nate, if you don’t already know it, Gadamer takes legal reasoning–common law reasoning, at least–to be an exemplary species of taste: it is knowledge and produces knowledge but cannot be reduced to a set of rules, concepts, and procedures that manipulate data.
I remeber one year in my Phd program, I took on class on using the scientific method in english studies. This method is used almost exclusively in the Rhetoric/Composition division. We read dozens of surveys and experiments that were geared towards figuring out which ways of teaching composition were the best, or how students understood the texts they read in a given writing class.
Near the end of the semeseter, I met with my professor and outlined my idea for a project. It used process tracing as a way to judge a certain strain of reader-response criticism that is still somewhat current in literary criticism.
My professor told me to start over from scratch, because I would never get it published. I asked why, and she replied that it doesn’t further any research in rheoric/composition and that the literary theorists couldn’t care less if someone was able to scientifically prove (or disprove) their pet theories.
Besides, with the strain of postmodernism most common at UT-Austin, scientific discourse is racist, sexist and classist anyway, so literary theorists would (though they may not use these exact words) consider it immoral on some level.
I find myself less and less interested in a PhD as time goes on.
Ivan Wolfe: Sitting on the promotion and tenure faculty of the College of Humanities at BYU, I’ve read a number of published articles in rhetoric and composition that took the kind of empirical approach that you take. Literary theorists wouldn’t be interested because your work isn’t literary theory. But I would be surprised if composition journals wouldn’t be interested.
Chris,
Well I agree with you and me, since it is apparent that we are saying the same thing :)
Frank,
Frank wrote: “Mathematics can be used to show that A follows B, but never that B is true.”
Depends on what you mean by “true”. We have to make all kinds of unproven assumptions every day just to operate in the world, starting with the assumption that you exist and that you can rely on the information your senses give you. So, really, in that sense you can never prove that *anything* is true.
I agree with your statement at one level — mathematics relies on a set of unproven axioms, and statements implied by those axioms don’t necessarily hold under a different set of axioms. But I don’t agree that that makes statements of truth in mathematics any less true than empiracly verified facts.
Frank also wrote: “We know how fast loads accelerate given a certain amount of force because we’ve tested it to death.”
True, of course. My point, though, is that we can take a basic scientific principle like that and apply it to very complex situations that have never been tested at all. I can very precisely calculate the motion of multiple bodies orbiting Pluto, even though I’ve never experimentally verified a similar situation. So, essentially, Newton’s Laws are being treated as axioms, albeit experimentally verified ones, and the resulting predicted motion of a complex system is a logically verifiable statement with no need for experimental validation. Assuming inerrant mathematicians, of course.
Ivan Wolfe wrote: Besides, with the strain of postmodernism most common at UT-Austin, scientific discourse is racist, sexist and classist anyway, so literary theorists would (though they may not use these exact words) consider it immoral on some level.
I find myself less and less interested in a PhD as time goes on.
In these sentences you have come very close to my own feelings about the abuse of theory in postmodern academics. Jim provided me with a very helpful reply as to postmodernism, including links to some of his essays on the subject, but these alone could not offset my larger concerns with the hijacking of academia by certain disproportionately loud special interests/pet theories.
Jonathan Greene soundly thumped me over there as to the true value of a PhD in humanities-related fields, while at the same time justifying a very exclusive attitude towards knowledge generally.
But your point here raises an interesting question that I have often pondered. Does the work product of people with PhDs in the “hard sciences,” fields in which scientific method and discourse is essential, make a more useful contribution to knowledge than that of PhDs in the humanities? A humanities/literature enthusiast myself, I grudgingly have to confess that I believe they do. My brother-in-law who has just completed an MD/PhD has a vast well of much deeper and more complex, methodological knowledge, wearily acquired through years of pain-staking adherence to the scientific method and to studying the margin of error in his own research, than could any humanities-related PhD possibly have, in my opinion. True, this might be merely comparing apples and oranges, but regardless it seems like a well of more useful knowledge precisely because acquired through the scientific discourse that is disparaged in the soft sciences.
You wrote: My professor told me to start over from scratch, because I would never get it published. I asked why, and she replied that it doesn’t further any research in rheoric/composition and that the literary theorists couldn’t care less if someone was able to scientifically prove (or disprove) their pet theories. That is an enlightening exchange and one that seems to me to reveal the problem with what is going on in literature and the humanities. Not only is it not possible to prove anyone wrong in these fields (outside of picking apart the internal logic of the argumentation), but the people spewing these theories and the supposed proofs of them (“Goethe and Why He Was Gay” etc.) deep down do not care whether the theories are provable or not, as long as they get them “out there” in print. As Jonathan Greene points out, margin of error or not, as long as the writer of the theory/application of a pet theory has a PhD, it must be addressed in any subsequent treatment. So what is the true value of such a spurious contribution, if the only “ripple” it makes is that it must be debunked by the next writer before moving on with an equally indifferent and unprovable pet theory? Is there any other way to approach humanities research and contributions to knowledge than the One True Way approach of Frank’s man with one watch?
Jim:
I wasn’t complaining that it is not possible to empircal research in rhet-comp. I was bouncing off this statement by Frank:
“What is frightening is when one sets academics or anyone loose to start offering theories which cannot be pinned down by any data. These are people with one watch. They have a theory; there is either no attempt to confront the theory with data or there is no ability to do so. The proffered attempts to prove the theory are laughably bad in their inability to actually prove or disprove anything. So instead of recognizing that the theory is unsupported, the theory becomes the One True Way because it is unrefuted. “
I know that there are ways to get empirical research in rhet-comp published. My problem was that I study literary theory and I am constantly bothered when literary theorists make claims that can be empircally tested, but refuse to accept any empirical testing.
It’s one thing to see if sentence combining produces better writing among college freshman than using writing models. It’s another to argue about whether Milton was of the Devil’s party without knowing it.
But when I wanted to use process tracing to see if certain claims (about how readers read texts) made by reader-response or pop culture theorists can be verified, I find that theorists could care less and even find the idea offensive.
John Fowles asks: “Does the work product of people with PhDs in the “hard sciences,” fields in which scientific method and discourse is essential, make a more useful contribution to knowledge than that of PhDs in the humanities?”
The products of technology ought to be useful. Perhaps some disciplines ought to be useful. But usefulness is a dangerous criterion for the value of a discipline. To be useful is to be good for something, i.e., for something else. The best things are not good for something else; they are good in themselves. If nothing is good unless it is good for something else, we find ourselves in an infinite regress where, in fact, nothing is really good.
I’m not much impressed with a good deal of what goes on in English and literature departments. There is, unquestionably, a lot of nonsense, and junior faculty have to put up with it when senior faculty, like me, can just roll our eyes and go about our business. But I doubt that we would improve those departments if we made usefulness a criterion of their existence.
(By the way, academic freedom originally meant the freedom to be useless, not useful to the purposes of the university’s sponsor.)
Jim, as one in the field of English, all I can say is “amen.” I still have some hoops to jump through before I can get to the eye rolling stage, but I’m looking forward to it. The problem with the current state of English is that it has stopped being useful OR good in itself. Literary criticism is a joke these days. I highly recommend “The Rise and Fall of English” by Robert Scholes for anyone interested in how to save English. Its probably applicable to the humanities in general.
Robert Scholes was an eye-opener to me. It’s because of his book I began to focus more on rhetorical criticism rather than the standard Postmodern and/or Marxist based theory that composes the majority of Literary studies today.
Jim: point well taken with your critique of my questions about usefulness. I agree with you.
But you also agree with me about the nonsense that constitutes these soft fields and as Ivan Wolfe pointed out, these people do not care about the provability–and hence the existence of a margin of error or disclaiming such–of their ideas. That is where I was going, in the spirit of Frank’s original post.
To eliminate the distracting question of usefulness, let’s put it this way: can anyone in any of these fields have two watches? Or is it simply in the nature of these fields, particularly in the postmodern climate, that theories are concocted by people with only one watch on the basis of the One True Way (or actually to be more consistent, No True Way).