Scientists are understandably concerned that their work be evaluated by their peers – not by those who lack the education or experience to distinguish between fact and surmise, between nuanced understanding and misleading sleight of hand, between credible analysis and superficiality. Are the peers also competitors? Might the peers be conflicted? Scientists still want to take their chances. They like their odds. It’s not unlike the spirit of that pickup basketball game discussed in an earlier post.
But there are other audiences whose opinion matters, whose review counts. Some might say that the public’s rating, whether educated or not, whether discerning or not, is the definitive word. There can indeed be wisdom in crowds. Lawyers and politicians speak of the court of public opinion. Isn’t there a kernel of truth to this as well? Certainly this applies in the free marketplace for goods and services. Think Apple products: the iPod, the iPhone, the iPad. You’ll happily pay more for a BMW than you will for a Yugo. Think the Miami Heat versus the Wizards. Or the Packers versus the Redskins. Think Lady Godiva chocolates versus a Hershey’s Kiss. Ideally, the political process leans that way, as in “the people have spoken!”
Here’s another, closer to home: a family member. What kind of reviews did you get from your parents growing up? Could you do no wrong? We call you spoiled. Could you never measure up? Psychologists find rich scope for work with adults from such backgrounds. How about your spouse? What she or he thinks or says matters, doesn’t it! My own wife is extraordinarily loving and supportive, but sometimes that support takes the form of criticism. And usually what troubles me is not that there’s so much of it, but rather that so much of it is on target. She’s right! Though she’s not a scientist, she has a many valuable insights on my professional life.
Edging a little closer still – what about our self-evaluation? This human habit or trait is at the core of why peer review works in the first place. Go to any university, and you’ll find faculty exhorting their students and colleagues: “be your own best and sternest critic.”
Well said! Certainly, at the end of the day, we have to “be able to live with ourselves,” “face ourselves in the mirror,” (insert your favorite aphorism speaking to your self evaluation here). This applies pretty broadly – to the whole of our lives. In Hamlet, Polonius tells Laertes:
“This above all: to thine own self be true,
And it must follow, as the night the day,
Thou canst not then be false to any man.”
But our self-evaluation, though it matters most in one way, matters least in another, doesn’t it? Certainly we should aspire not to delude others or ourselves. And before asking anyone else to spend time checking our work, we should prove to our own satisfaction that it merits others’ scrutiny. Nevertheless, it is easier to encourage others to follow this practice than adopt it ourselves. And behavioral scientists tell us we are statistically far more approving of our feelings, thoughts and actions than those behaviors merit.
Fact is, we play to all these audiences, don’t we? And what these multiple and diverse audiences think – how they see us – matters a great deal. If they approve, we feed off the energy. If they scorn our work, our entire lives seem to lose much of their meaning and purpose. So it’s quite tempting to be defensive and protective of our accomplishments, to be more interesting in winning the debate over the correctness of our publications or actions than in seeing whether we deserve to win that debate or not.
But for Earth science – science concerning the natural world – there’s one more audience we haven’t mentioned. And for Earth science, this may be the one audience that matters most … the real world. Earth scientists purport to study the way things are. This real world has no interest in, or capability for, spin, or deception. It does what it does. It is what it is. We remembered Isaac Newton in the previous post. Why do we think him special? Because everywhere anyone looks, his laws seem able to predict what the universe will do next. A rather remarkable and extensive edifice of understanding has been built on his work. The bumper sticker says: “Gravity. It’s not just a good idea. It’s the law.” Lie in the grass in an orchard today, three hundred years later, and an apple will fall on your head.
So when you or I, or anyone for that matter, writes a paper, or runs a computer model, or makes field measurements, or interprets the same, we ought to be looking past our human audience, however extensive or small, and looking deep into the core of the natural world. We ought to be asking “how does my work or my idea or my measurement square with the real world?”
Most Earth scientists do this. But given the pace of today’s life, and the rewards for success in the eyes of others – the tenure, the grants, the promotions, the visibility – it is tempting to take a shortcut. It is so easy to relax the standard a bit, and to compare our theory or hypothesis not with reality, but with other people’s opinions, or representations or models of that reality. It is tempting to compare our paper not with fact, but with others’ similar papers – to substitute a virtual reality for the real thing. We can be seduced into comparing our undertakings not to what is needed by a world hungry for environmental information, but only to the efforts of others in our small community.
Ultimately, it’s not who wins the climate change debate that matters – it’s what the climate actually does, and how any variability or change impacts human affairs. Those who judge the debate can easily be misled, can be wrong. By contrast, what the climate actually does allows for less argument. Will the Arctic be ice-free? If so, how soon? Will Siberian tigers be extinct before the end of this century? The real world will let us know. One side or the other can “win” such debates, whether in public, or the scientific literature, hopefully most of the time through merit, but sometimes through sheer force of personality, or education, or debating technique – or politics. But frankly, there’s no longer time for that. The signs are, our human predicament on this planet is more urgent. We don’t need intellectual virtuosity so much as we need insights to guide pressing real-world decisions.
So, let’s all examine peer review, and do what we can, both personally and in community, to strengthen it. But perhaps we shouldn’t do that over much. Perhaps we should settle for a little bit less tight peer review in favor of gaining a lot more real-world experience with our science and technology as we develop them.
More on this topic in the next post.
Pingback: The Earth Whisperer. | Living on the Real World