An earth observations dream gone awry.

“Dreams are never concerned with trivia” – Sigmund Freud

“Yet it is in our idleness, in our dreams, that the submerged truth sometimes comes to the top.” – Virginia Woolf

My dream last night featured EarthOracle, an entirely hypothetical e-business collecting and offering Earth observations – in short, weather data.

Things started innocently enough. In that dreamworld, EarthOracle worked with vendors to provide surface weather-observing kit to anyone interested in setting up a station and contributing data from his/her location to EarthOracle’s pool. EarthOracle bundled and sold an array of tailored data streams to governments and to private sector users of the data – trucking, rail, and aviation; agriculture; utilities; water resource managers; any and all who had use for local observations. Those citizens operating the observational sites received payback from EarthOracle based on demand for observations from their particular locations. For most, including EarthOracle itself, it was initially a money-losing proposition, but there was the pride of participating in crowdsourcing for societal benefit.

Time passed. A few individuals, sensing opportunity for enhanced returns, went through the work and trouble needed to establish small subnetworks of observations providing coverage for areas they saw as having special commercial potential. These weather-data collectors were content to pay EarthOracle overhead. As they saw it, EarthOracle was performing a valuable function in aggregating the data and then marketing and brokering them to governments and companies worldwide – a kind of PayPal for Earth observations. About the same time, EarthOracle found itself ready to branch out – to deliver CASA-like radar data covering urban areas and gaps in the conventional U.S. weather-radar coverage. In offering such additional services, EarthOracle was starting to eat from the rice-bowl of established-but-small private-weather-service providers.

Venture capitalists began to take notice. With the additional cash inflows, though still losing money, EarthOracle was in a position to launch cubesats – small satellites weighing only a few pounds, and costing less than $100K per unit, that could be placed into orbit in large numbers and provide global observations from space. The new data streams seemed insignificant in the scheme of things – most satellite observations were still being provided by the big government-owned-and-operated polar orbiters and geostationary satellites. But it wasn’t long before EarthOracle comprised a small but vibrant ecosystem of private-data providers.

EarthOracle leaders then expanded their vision. They knew that for most locations at most times, the additional data EarthOracle had on offer provided only minor incremental value to numerical weather prediction (NWP) and forecasts. Often the weather outlook itself would be benign, so that uncertainties in those forecasts of the kind that could be reduced by more data were themselves small and didn’t matter much. Sometimes the weather outlook would matter, but the EarthOracle data were either of the wrong type or from the wrong locations. But sometimes, and for some places, EarthOracle data could make a consequential difference in the forecast – increasing accuracy, reducing uncertainty, and adding significant utility to impact-based decisions with respect to some high-dollar-value, weather-vulnerable circumstance. For any given spot, such conjunctions would be unlikely. However, given Earth’s 200 million square miles of surface area, on any most days a handful of such locations would be in play somewhere worldwide.

Their customers began to catch on. Use of EarthOracle data was on the rise. EarthOracle balance sheets showed new strength. For the first time, building assets and positive cash flow could support further innovation.

That’s when the EarthOracle CEO had the aha! moment. She realized that the ensemble forecasts in wide use by weather services across the world didn’t simply identify areas of forecast uncertainty at critical places and times (regarding the details of genesis, path, timing, intensity and duration for landfalling hurricanes, winter storms, major tornado outbreaks, or flash floods). The model runs also pinpointed regions upstream from these events where lack of data contributed most to the forecast uncertainty. Moreover, the relevant data-sparse regions could sometimes be determined days in advance – providing enough time to collect additional observations adaptively. Accordingly, EarthOracle developed and deployed fleets of instrumented aircraft and drones that could be flown on short notice into the areas where supplementary observations would add value. EarthOracle could use quick-and-dirty model runs to spot areas of need even as they were beginning to attract the attention of conventional weather services, private and public. EarthOracle would then nimbly collect high-value data, and again run realtime NWP to confirm reductions in forecast uncertainty in response to the added proprietary data. They could then confidently promote and market these.

EarthOracle’s value proposition to its government and private-sector customers exploded. As the success stories multiplied, and were widely reported, EarthOracle usage soared. Worldwide, national weather services and companies couldn’t afford to be denied the additional data, and besides – they were cheap. What’s more, governments and companies hadn’t had to foot the bill for the instrument platforms; they only had to purchase the data streams.

My sleep was troubled. I began to toss and turn… but continued to dream on…

At this point EarthOracle leadership, flushed with success, went a step too far – introducing surge pricing[1].

Most of the time, and for most places, EarthOracle would make its data available for a nominal fee. But when EarthOracle could identify instances of special public or commercial concern, combined with forecast uncertainty, they’d use their adaptive observing capability to capture needed data of high-value, and then sell it only at prices several times above acquisition cost.

Both government- and private-sector users now found themselves caught between a rock and a hard place. EarthOracle had demonstrated in specific past instances that it had skill in identifying the value in its data. It wasn’t bluffing. Governments had no in-house capability for taking such adaptive observations; the money and the investment history weren’t there. In the early going, they’d found it seductively easy and inexpensive to simply purchase the data on offer. EarthOracle’s capability had even been welcome; it had saved government agencies from major capital costs Congress was in no mood to incur.

But now government had to meet EarthOracle’s price or fall short of delivering the best-available weather forecasts for protection of life and property. And thanks to extensive and growing media coverage of weather events, all this was becoming public knowledge. EarthOracle’s market value skyrocketed – exceeding one hundred times earnings. An initial public offering (IPO) in the works that looked to value EarthOracle at over one hundred billion dollars.

Then came the day when, one week out, it was clear that a major nor’easter – a 500-year event) was going to hit the east coast, somewhere between New York City and Boston. EarthOracle offered a data set it said would pinpoint the area of greatest risk – asking a cool $500M. At the same time, a second storm event that had been poorly forecast was unexpectedly hammering the California coast, producing flash floods, triggering landslides, and causing major loss of life and property damage. No analogous EarthOracle data had been available. The reason came to light: EarthOracle management, lacking the assets to meet the data needs for both the east- and west coast forecasts, had made a choice just days earlier, favoring the east coast, on the basis it would maximize profits. Congress and the public were coming unglued; picketers blocked the entrance to EarthOracle’s headquarters…

…I awoke with a start, in a cold sweat. Whew! – only a nightmare!

Of course, none of this could ever actually happen. This was a mere bad dream. No reason for concern or thoughtful reflection by anyone living on the real world.

_______________________

[1] Some may be unfamiliar with the term or how surge pricing works. For the car-ride-for-hire firm Uber, here is the background, taken from the Wikipedia article:

Uber uses an automated algorithm to increase prices to “surge” price levels, responding rapidly to changes of supply and demand in the market, and to attract more drivers during times of increased rider demand, but also to reduce demand. Customers receive notice when making an Uber reservation that prices have increased… The practice has often caused passengers to become upset and invited criticism when it has happened as a result of holidays, inclement weather, or natural disasters. During New Year’s Eve 2011, prices were as high as seven times normal rates, causing outrage. During the 2014 Sydney hostage crisis, Uber implemented surge pricing, resulting in fares of up to four times normal charges; while it defended the surge pricing at first, it later apologized and refunded the surcharges. Uber CEO Travis Kalanick has responded to criticism by saying: “…because this is so new, it’s going to take some time for folks to accept it. There’s 70 years of conditioning around the fixed price of taxis.” Uber released a post detailing why surge pricing is in place and how it works. They emphasized that without surge pricing, Uber would not have its trademark service of pushing a button and getting a ride in minutes. This is detailed in a case study around a sold-out-concert at Madison Square Garden when surge pricing took effect. During this event, the number of people who opened the app increased 4x, but the actual ride requests only rose slightly, enabling ride requests to be completed with the usual ETAs.

Surge pricing makes supply and demand match so efficiently that the waiting time is almost always below five minutes, regardless of the circumstances. Surge pricing increases economic efficiency in two ways: 1. rising prices motivate more drivers to start driving, 2. when there are not enough drivers for everyone, the rising prices make only those customers accept a ride whose needs are highest.