Wednesday 24 August 2016

Ich mach mir die Welt, wie sie mir gefällt





Das P-Hacking-Dashboard von FiveThirtyEight - eine wunderbare Demonstration der Manipulierbarkeit von Erkenntnissen: http://projects.fivethirtyeight.com/p-hacking/

http://projects.fivethirtyeight.com/p-hacking/





Wednesday 19 June 2013

Noisy Politics

As elections approach in Germany, the number of surveys on voting intentions increases. Every week you can read some wild speculations on why party X lost one point, while another gained one. If you are tired of journalists who interpret nothing but noise in the data, you might want to have a look at the diagrams below. They show you the long term trend in voting intentions for each major party as a moving average of individual survey results (including surveys from all major research agencies). Confidence intervals for every survey estimate are also shown. The data is up to date - last included survey was published on June 18.






Monday 10 June 2013

The “yes we can” fallacy of market research



http://ecx.images-amazon.com/images/I/41bVTTI7mTL._SL500_AA300_.jpgThe plenitude of tools and specialisations that agencies offer to potential clients on the research market is stunning. If you listen to marketing speeches of the research industry, you will be tempted to believe that any question can be answered by research, at any time, anywhere, for any target population, and at reasonable costs. You want to know if the packaging of your product should be gradually bluer? You want to measure the long term return on investment of hiring external head hunters with absolute precision? You want to estimate customer satisfaction with a service that does not even exist yet? You might even have more esoteric issues that I cannot imagine right know? Just ask a research agency about the feasibility of your research project. The answer which is most likely to get is: yes we can. This might be the consequence of a competitive market. If one agency says “no we can’t” a potential client easily goes to another one. However, this does not mean that the conclusions drawn from overstretched projects are any better than speculations. But naïve trust in numbers makes dubious research a profitable undertaking for some agencies. Actually, many clients do not buy knowledge and insights but ‘numbers’ - no matter how they came about. To some people ‘numbers’ have the aura of precision and truth, at least as long as they broadly fit into one’s own presumptions on an issue. This is surely a poor understanding of what research is good for, but a popular one nonetheless. Basic methodological issues like sampling, statistical inference or psychometric considerations in research design are ignored way too often in practice. Ignorance rules in many businesses which buy research services. So, if you are on the client side and if you really want to gain insights, than focus on methodologically justifiable projects and give up on projects that are likely to produce nothing but useless ‘numbers’. And if you are really in the need for ‘numbers’, just throw some dices. This will deliver equivalently useless ‘numbers’. But it is significantly cheaper than commissioning research.