MDB Insight Blog


Statistical Validity (and coffee)

#WednesdayswithMDBInsight – Our Wednesdays with MDB Insight posts feature the thinkers and doers on our team sharing ideas and talking about what’s important to us as professionals. We have very diverse backgrounds and a range of interests to share with you. We hope you’re enjoying these posts and that you will join the conversation with us and let us know what’s on your mind mid-week. This week Chris Bandak, EVP Market Research & Analytics, discusses statistically valid surveys and coffee.

Chris Bandak, EVP Market Research & Analytics

There are two things I’m particularly known for among my colleagues. The first, of course, is my love of all things market research, and my keen interest in applying new and improved data analytics to the most pressing issues facing communities today. The other is coffee. I get teased about being a connoisseur (a title I’m not entirely comfortable with) but it’s true that I take my coffee seriously. So, when we decided to start these regular Wednesday blog posts, we thought it would be fun to combine those two things into a coffee break with a side of what’s-up-in-market-research.

One of the questions I get asked regularly about market research is this: “the population of my town is 13,790 people, so how many survey responses (completes) do we need if we want a statistically representative sample?”.

I really like this question because it reinforces the idea that statistical validity is something to build into your surveying. For anyone using market research to help make decisions, the notion of having a statistically valid sample is important for several reasons.

  • First, the sample can be used to represent the views of the population at large;
  • Second, the results take into account the opinions of everyone and not just those with polarized opinions;
  • It also makes it possible to run significance testing, state a margin of error, and other statistical models (the profession doesn’t allow for these where a sample has not been drawn randomly from a population).

Many people are surprised to learn that using the same sample size for a population of 5000 or 22,000,000 produces essentially the same margin of error. In other words, a poll of 500 people across the entire population of Canada has the same statistical accuracy as it does for a small town with a population of 10,000. So, what sample size should you use? This comes down to the level of accuracy needed in the research you are conducting. Large sample sizes have lower margins of error and vice versa. If the goal is to compare the effect of a new drug against a placebo you would want a larger sample size. If the goal is to get a general sense of community support for a new policy, you could use a smaller sample size. The second factor that will determine the sample size needed is the number of segments you need to analyze at the conclusion of the research. The more segments you require the larger the sample size needed. To find out how many people in Anyville like coffee, it would be ideal to ask all of them. But that’s not feasible, so we need to ask a representative sample of the Anyville population large enough to give us a statistically valid result. If the population of Anyville was 500,000 we would sample a group from 250-400 people, making sure to use quality research standards and a methodology where everyone in the population as an equal probability of being selected. What if the population of Anyville was 10,000,000 people? We would still only need to ask 250-400 people. A larger population count doesn’t mean we need a bigger sample size to generate a statistically valid result.

Chris’ Coffee Pick:

mocha-2111932_1920For me, there’s nothing quite like the experience of drinking Turkish coffee. It has a unique taste and robust flavour, but beyond that it’s the whole presentation that I find enjoyable. It’s steeped in long held tradition and encourages taking time to enjoy the coffee along with the company. The little cups, the ornate designs on a silver tray, even the decorated spoons seem to demand that we slow down and savour every sip. UNESCO has even added Turkish coffee culture and tradition to its Representative List of the Intangible Cultural Heritage of Humanity.

If you haven’t tried a Turkish coffee yet, find a spot and treat yourself. If you’re lucky, there will be someone to read your fortune from the residue in your upturned cup afterwards!

Chris 2018Chris Bandak is Executive Vice-President Market Research & Analytics at MDB Insight. To learn more about Chris and the rest of our team, go to



Related Posts