Paul Saffo is a lauded forecaster and essayists on the future of technology. He’s part theorist, part historian, and part story teller. Saffo is associated with several prominent organizations, including Stanford University, Institute for the Future, and The Long Now Foundation. His talks and writings are consequential, timely, and refreshing. I am a big fan of his work and of The Long Now Foundation.

On January 11, 2008, Saffo spoke on “Embracing Uncertainty: The Secret to Effective Forecasting” at one of Long Now’s Seminars About Long Term Thinking. Having recently relocated to Boulder, Colorado from the San Francisco Bay Area, I wasn’t able to attend this live, but I did enjoy the audio repost and I’m looking forward to the video.

Saffo discussed several of the methods he uses in forecasting, which he also recently described in the Harvard Business Review article entitled “Six Rules for Effective Forecasting.” He started out by recalling one particular forecast he received from a friend some time ago which was supposedly from CNN, although Saffo was been unable to confirm it’s true source. The forecast read, “Hunt for Bin Laden: Experts Agree Al Qaeda Leader is Dead or Alive.” Saffo remarked that on the surface this was not a good forecast, being quite obvious, but in fact that is was actually a great forecast, because it accurately and completed captured the uncertainty of that moment. He said “The biggest mistake of a forecaster can make is being more certain than the facts suggest.” I suppose that assumes a high degree of confidence in the validity of our perceptions / beliefs, and a high degree of actual validity in those perceptions / beliefs, but that is an entirely separate nut to be cracked at another time.

Saffo said that “Uncertainty is not just an artifact of imperfect foresight. Uncertainty is intrinsic in the process and in my opinion, it is very good news. That’s good news because uncertainty is opportunity.”

I’ve pulled out some nuggets from his talk, but the crux of it was that uncertainty is everywhere and that forecasters must embrace it.

Embrace uncertainty

How does one embrace uncertainty? Saffo suggests that you must “map the cone of uncertainty” by taking an event and extending outwards from that event. It’s a cone shape because the degree of uncertainty increases over time. I know what the weather is like right now, I have lower confidence about what it will be like fifteen minutes from now, and I would have extremely low confidence in describing the weather in two weeks. This seems very obvious, and Saffo remarked that much of these forecasting best practices are common sense and so not terribly original. What’s important about the cone is not the shape, but how widely or narrowly it is drawn, and the difficulty one has in describing the edges. He went on to say, “The art of forecasting is understanding uncertainty and also balancing between a stance where if you look at things to narrowly, if you draw that cone to narrowly, you’re going to miss things that happen, if you draw it to broadly, you’re going to spend your whole time navel gazing around events that may never come to pass. The art, what makes forecasting hard, isn’t predicting the outcome. What makes forecasting hard is predicting the edges of the cone.”

Never mistake a clear view for a short distance

Saffo said that “things surprise us for a very simple reason and that is that change is never linear.” I think that Nissam Nicholas Taleb may argue that things surprise us for different reasons, but that too is for another time. Saffo suggested that effective forecasters are constantly looking for s-curves, the model of so many technologies and trends that we see today. An s-curve (sigmoid curve or logistic curve) is a model of the growth of something where growth begins slowly at first, then is characterized by an inflection point followed by exponential growth, followed later by slowing growth and finally zero growth. S-curves often stack on top of each other, which many believe results in a true acceleration of change. Saffo is skeptical of whether or now we are experiencing a true acceleration, but acknowledged the phenomenon of compounding s-curves, especially in technology.

“The future constantly arrives late and in unexpected ways.”

Saffo argued that most “overnight successes” are really ideas that are about 20 years old at the time of their success. He provided several examples, including the demise of LucasFilm’s Habitat and the successive failures of similar products which eventually lead to Second Life. He provided another example of the takeoff in web technologies resulting from how an oversupply of engineering talent in Silicon Valley following the failure of interactive TV in the early 90’s. He suggested that if you want a short term win in the market, look for something that has been failing for 20 years and everybody says will never happen – invest in that. “[Silicon valley] is built on the rubble of failure, not the spires of earlier success.”

Look for indicators and things that don’t fit

Saffo said the way you avoid being blindsided is to look for indicators. He mentioned research published in 1977 suggesting global climate change, Usenet newsgroups in 1984 discussing a software bug that would affect computers in the year 2000 (Y2K bug). He discussed the Roomba and how research showed that 2/3 of Roomba owners named theirs and 1/3 had taken them on vacation or to a friends house. These are indicators. IFTF calls them weak signals. The success of the 2007 DARPA Urban Grand Challenge at virtually the same time that a 108-car pileup occurred with human-drivers is another indicator. Saffo argued that forecasters should look for things that don’t make sense or are just really weird.

 

Saffo commented that Peter Schwartz, “a brilliant futurist, is fond of remarking that the difference between a good forecast and reality is that a good forecast has to be believable and internally consistent – and of course, reality labors under no such constraints.” The moral of the story is that it is important to look for wildcards. Saffo said that wildcards “test the edge of the cone” and “define the ragged edge of plausibility of any good forecast.” Several places to look for wildcards are in really bad forecasts, bad magazines, and science fiction.

Look Back Twice As Far As You’re Looking Forward

Historical observation can provide hints as to what may happen in the future. It’s not surprising that many of the best practitioners of foresight are historians. Saffo certainly is no exception. Saffo said that “a rear view mirror is a damn good forecasting tool if you use it in the right way… backsight is the secret to foresight… look back and random little indicators will suddenly line up into a very powerful beacon hinting at the future.”

Be indifferent

Separate your preferred future from the future that is most probable.

Assume you are wrong and forecast often

“Good forecasting is the inverse of traditional good research” where conclusions are reached only after carefully analysis of data. Instead, come to a conclusion early and set out to prove yourself wrong.

Additional Saffo quotes to enjoy:

  • “Embrace uncertainty. In all of its complexity and gut-wrenching portent of change, uncertainty is our friend, uncertainty is opportunity.”
  • “I don’t predict, I forecast, and that’s about mapping the cone of uncertainty.”
  • “When change clusters at the extremes, it’s a certain bet that much more fundamental change lies ahead.”
  • “We fail our way into the future.”
  • “Wildcards sensitize us to surprise.”
  • “Every decade or so, an enabling technology arrives that sets the entrepreneurial landscape.”

On Feb 4th, 2008, Nassim Nicholas Taleb will be speaking at The Long Now Foundation about his book Black Swan, which I am eagerly anticipating.