Collaboration in Earth observations, science, and services

The previous post tables the subject of public-private-sector collaboration. Jerome Ravetz cites UK examples showing that such relationships are fraught. He asks whether , when it comes to such collaboration with respect to Earth observations, science, and services, we find the situation to be any better…and if so, why?

As noted then, people in our field often claim that the Earth sciences, unlike physics and chemistry, are necessarily place-based and inherently cooperative. Take weather forecasts (and we’ll focus only on that one example here). Does the world want global weather forecasts, good out to several days? Then nations need to share their data on current atmospheric conditions.

But where does the public-private-sector piece enter in?

It turns out that such collaborations between governments and for-profits date back to the earliest days of national weather services…roughly speaking, to the latter half of the 19^th century. They owe their origin not to any particular scientific insight but rather to the invention of the telegraph.  Meteorologists had known for some time prior that weather formed patterns, and those patterns evolved as they moved over the Earth’s surface. But when weather information could be sent from one location to another only by letter, those weather patterns and their movement could be discerned only days after the fact. However, as soon as telegraph operators started entering weather information as part of their regular hourly transmissions, it was possible to assemble a picture of the weather across a country as big as the United States in real time, and track the movement and development of storms.

In the United States, in the years following the Civil War, telegraphic infrastructure was largely in the hands of Western Union, a private firm. The U.S. Army Signal Service was responsible for the forecasts. Throughout those early years, the Signal Service was constantly negotiating with Western Union – seeking priority for weather information at the regular intervals needed to assemble the synoptic weather data, and seeking to lower the cost and burden of such data transmission by developing a shorthand of codes and abbreviations. [Bandwidth was a problem, even for the Victorian Internet.]

Reading this history, one finds unsurprisingly that the negotiations pleased neither party. Signal Service officers felt Western Union wasn’t sufficiently deferential to a national imperative and the paramount public good. Western Union officials sensed the Signal Service lacked respect for their need to make a profit.

This, in a nutshell, was the Jerry Ravetz problem.

With the development of broadcast radio and television, an additional dimension was added: private-sector dissemination of (then) Weather Bureau forecasts. [About 1890 the weather-forecasting function was moved out of military control into civilian hands; the Weather Bureau was placed in the U.S. Department of Agriculture. In 1940, in acknowledgment of its broader role in the U.S. economy, the Weather Bureau was moved to the Department of Commerce, where it remains today.] By the end of World War II, a private-sector forecasting industry had emerged, and was already chafing under what it occasionally saw to be unfair competition from the Weather Bureau.[1]

Space doesn’t permit a detailed history here but the ensuing sixty years have seen expanding public-private collaboration, and interweaving of the respective roles. For-profit aerospace firms build observational systems such as satellites, radars, and networks of in-situ surface sensors for sale to the federal government. Other firms sell data streams to the government, such as data on lightning-stroke location, carbon dioxide levels, etc. The private sector delivers weather forecasts to the public by broadcast and internet. Private-sector firms have developed and market a wide range of forecast products tailored for proprietary use by airlines, rail, marine transportation, agribusiness, utilities, insurers, and many other sectors.

For most players and at most times, cooperation has been and is the rule, but there have been trouble points in the relationship. National Weather Service forays into internet services and social media have precipitated discussion. Official predictions of imminent threat to life and property such as tornadoes and hurricanes have been held to be the province of the public sector, because of a perceived need to put out “a single official message” and because of liability issues. In recent years, private-sector firms have noted that social media and informal networking with respect to weather hazards make this boundary less sharp and more problematic going forward. The business model of every-decade-or-so-multi-billion-dollar winner-take-all procurements for weather satellite and supporting ground systems has proved problematic for both industry and government.

In 2003, after two years of study, the NAS/NRC published a landmark report addressing these issues: Fair Weather: Effective Partnerships in Weather and Climate Services. Among its several recommendations was an unusual one…a suggestion that a specific, named organization – the American Meteorological Society – might provide a useful, neutral venue for the private- and public-sector providers of weather and climate services to discuss issues relating to the public-private partnership. The community followed this advice. The initial conversations were awkward, but today an AMS Washington Forum held every April, an AMS Summer  Community meeting held every August, and AMS Annual Meetings, held in the winter months [supported by ongoing working-level discussions that run throughout the year] have greatly improved the collaboration, not just to the benefit of the parties involved, but to the American public.

But challenges lie ahead. To go back to Jerry Ravetz’ question…why have meteorologists and climatologists fared so well?… the answer lies not only in the inherently cooperative nature of the work, but also in the fact that the collaboration has been low stakes (amounting to a mere few billions of dollars per year against a $14trillion U.S. economy and a $60+trillion world GDP), and therefore low visibility. It doesn’t take too much foresight to realize that this is changing. In the 21^st century, extraction of resources from the Earth (food, water, energy, and more), protecting the environment, and building resilience to natural hazards are growing more noticeable and problematic. There’ll be more money to be made for those who get things right. Those who get things wrong will pay an ever-higher price. This will attract more media visibility and provide a wide range of observers opportunity and incentive to find fault.

Furthermore, public-private collaboration in providing weather and climate services is different in every country. A few illustrations. In the United States, the government freely provides raw data to the private sector. In New Zealand, the weather service is itself a private-sector entity. In Europe, a landmass roughly the size of the United States has more than a dozen national weather services. The understandable result? In Europe, cost-recovery by sale of data is a key pillar of each weather service’s business plan. Europe and America are therefore no more plug-compatible in their provision of weather and climate services than  they are with respect to their electrical outlets. And these differences fray the fabric of the international cooperation that has been the hallmark of meteorological services worldwide until now.

The history of public-private collaboration continues to be written. The most interesting chapters lie ahead.