How much (valid) research is in market research?

20 April 2007

“How independent are independent analysts?”, asked Duncan Chapple in his post on Analyst Equity. Another interesting question often discussed at DARA meetings is: How professional and valid is the research offered by market researchers? Without any doubt there is a lot of reputable research available but the current hunger for market statistics also nurtures a trend for “quick & dirty” research – which can be observed at both big and small vendors. As there is no authority to set and control quality standards in market research users need to be able to judge for themselves and know how to separate the wheat from the chaff.

 

Interestingly enough an analyst, Andreas Stiehler of Berlecon Research, has written a  spotlight analysis on “ICT market figures and their reliability”. Unfortunately it is only for readers of German. Although I have been told that it is now trendy in the English speaking world to speak German ;-)

 

Basically he points out what the main indicators for bad and good research are and what users – that don’t have a degree in statistics – should look for when examining the quality of market figures. As always, the best way is to use common sense and ask the right questions, e.g. can a survey among 100 people around the globe really tell something about a market? When you break it down on a country level the basis will always be too low to base any assumptions on it. How good are online surveys if the group I want to examine is known to be slow in adopting new technology? And if a survey was done “among 100 people”, does that really imply that all of them actually responded?

 

But he is quite aware – and I would fully agree here – that there is no perfect research and it isn’t necessarily a bad thing to base insight on a small number of respondents or work with online surveys. We all know that collecting and interpreting data is hard work and almost every data might be of worth for some purpose. As long as the user is given clear and full insight into how, where and by whom the data has been retrieved and where there are possible restrictions in interpreting the data.

 

If anybody is interested in a full English version I can go and ask Berlecon if they would supply one.

4 Responses to “How much (valid) research is in market research?”

  1. Duncan Chapple

    Thanks for the mention Heidi. I have translated Andreas’ article at analystequity.com;

    <a>http://analystrelations.blogspot.com/2007/04/ict-numbers-and-their-reliability.html&lt;/a&gt;

  2. Duncan Chapple

    Your blog didn’t like that, so try this:

    http://tinyurl.com/yv7dqu

  3. Dom Pannell

    Thanks for taking the trouble to translate the Article, Duncan. German is not one of the languages I speak and I’m grateful that I’m now able to read the article.

    The TinyURL works fine.

  4. ARcade

    I’m posting this with a degree of trepidation given the recent article by Berlecon analyst Dr. Andreas

Leave a Reply