ProClim bringt die aktuellsten wissenschaftlichen Erkenntnisse zum Klimawandel in die politische und öffentliche Diskussion ein. Das Forum vernetzt Wissenschaft, Politik und Gesellschaft und trägt zu einer klimaneutralen und -resilienten Schweiz bei.mehr

Bild: NASAmehr

The problem of climate opinion polls

The danger of misinterpretations of surveys

Wetter und Klima (Symbolbild)
Bild: NASA

In recent times there has been a growing number of surveys aiming to investigate the opinion of climate and other scientists concerning scientific (and political) questions about anthropogenic global warming. Some of these surveys are conducted by climate scientists, some by journalists, others by climate sceptics. The background of these polls mainly is an often heard assertion (mainly by sceptics) that there is no consensus among scientists concerning global warming and that an important part of the scientists do not agree with the IPCC report. One intention of such surveys is to shead light on (or support) these claims.

There are a couple of well-known problems with such surveys. Besides the choice of the respondents, there are crucial problems concerning the quality of the questions that are used. First, multiple-choice types of surveys, which are exclusively used, prescribe answers and therefore might be biased. They often do not allow the respondent to express his real opinion. A differentiated answer, which would be appropriate in the case of such a complex problem as climate change, is not possible.
Second, and even more problematic, most often the questions are far too general and simple and leave open, what exactly is meant by certain expressions. Therefore, a meaningful interpretation is not possible, because the understanding of the question will be different between respondents. Since the time of respondents is restricted, often the most appropriate answer is chosen, although it does not really represent the opinion of the respondent. There is a high probability that this leads (intentionally or not) to misinterpretations of the results. A sophisticated choice of possible answers also allows to provoke misleading results.

One example for illustration:
Question (translated from German): «To make climate computable, climate models have to be sufficiently precise. Is this condition a) today already satisfied; b) not yet satisfied, but satisfiable in future; c) not satisfiable in principle.»
This question is highly imprecise: what does «computable» (berechenbar) mean? At what state of knowledge is climate «computable»? Which aspects of climate (sensitivity, projections, attributions, etc.) is the question referring to? What error range is the limit for something being «computable»?
The interpretation of such a question will differ widely between respondents. Here, given the prescribed answers, most scientists will chose answer (b), because we all know that climate models include important inaccuracies and knowledge gaps. The answer certainly is correct. However, since «computable» has not been defined, any kind of conclusion can be drawn. The conclusion that has been formulated in this case by the authors of the study: «Most scientists currently consider climate to be incomputable and climate models to be inaccurate», and: «climate scientists do not feel confident». Is this really what the respondents wanted to say?

Thus, we advise a good deal of caution with such surveys. Questions which are imprecise and can be understood in different ways, probably should better not be responded to. We recommend instead to notify the authors of the survey of the problems of understanding and interpretation. One should bear in mind that the choice of only the most «appropriate» answer (if the right one is not available) in most cases leads to misinterpretations.

For some time now there is an ongoing public survey by the German journal «GEO» which has also been sent to climate scientists and contains a number of problematic questions.

A recent contribution on «realclimate» discusses another ongoing survey.

For questions and comments concerning this issue please contact

Kategorien