superforecastingIn September 2015 Phil Tetlock and Dan Gardner released Superforecasting: The Art and Science of Prediction. The book summarizes findings from about 5 years of the Good Judgment Project (GJP; one of five participants in the Aggregative Contingent Estimation program of the IARPA). The overall goal was to investigate the forecasts made by the U.S. Intelligence Community (a federation of 17 intelligence agencies, e.g., NSA and CIA) and compare it to forecasts made by lay people without access to classified information and done in their spare time.

For me the book includes 3 key learnings:

  1. a good introduction into the Brier Score: this scoring method measures the accuracy of probabilistic predictions by combining “Calibration” and “Resolution” of a series of forecasts
    1. Calibration is a measurement that compares predicted probability vs. actual outcome; example: if a meteorologists predicts 70% chance of rain will it actually rain in 70% of the cases (if it only rains in 50% of the cases it is called over-confident; if it rains in 90% of the cases this person is under-confident)
    2. Resolution is the decisiveness of a predictor, i.e., saying “this will happen” and it does
  2. Traits of people with exceptional forecasting skills, most importantly:
    1. number affine: often with a background in math, science, or computer programming
    2. thinking style: starting from the “outside” and only then working towards the specific question to avoid confirmation bias
    3. active open-mindedness: from the book – “for superforecasters, beliefs are hypotheses to be testet, not treasures to be guarded”
  3. Teams are better than individuals: predictions from people working in groups were about 10% more accurate than those working alone

If you do not want to buy the book here is a link to a great paper with the most important aspects on this topic by the same author.

Christoph recently gave a talk titled “Crowd Intelligence” with the following agenda:

  • About group decisions
    • Why should you ask a group?
    • What can/will go wrong?
  • Crowd Intelligence
    • Minimum requirements
    • Asking the right question
    • Interpreting results
  • Crowd Status in Action
    • Demo
    • In your department
    • Industry examples

Download here: Crowd IntelligencePresentation pdf

In June 2015 Prof. Dr. Peter Kruse died at the age of 60. He was one of the leading experts in change management and an internet visionary. His interdisciplinary work focused on the application of collective intelligence to economic and social developments.

A talk from Peter Kruse about decision making in complex situations was influential to us. He presented the following 5 strategies:

  1. Try and error
  2. Ignoring the problem
  3. Rationality
  4. Simplification
  5. Gut feeling

He argued that gut feeling is the best strategy we can apply today. However, we need to be aware that our instinct may be misleaded due to a quickly changing environment, i.e., a strategy that worked 5 years ago might not be applicable today. Therefore, we need a group and use collective intuition to avoid solutions from the past.

Below a YouTube video with this talk from Peter Kruse (in German only):