This is Part 3 of a 4-part series overview of market research, how to conduct and talk about research. It is designed for a general audience and not necessarily for trained or professional researchers. In Research 101, we talked about the basics of research and how it starts with a business decision. And in Research 102, we discussed the different sampling methods and what's right for your business. Today, we're talking about how important your sample is when doing research.
Mitigating Bias in Convenience Samples
In Research 102, we concluded that using a convenience sample is the most cost-effective and efficient way to gather survey responses in order to make an informed business decision. However, because they're not strictly random, we need to do certain things to make sure that we are mitigating bias.
1. Adjust the Demographics
If you know the demographics of your target population (based on U.S. Census or other secondary data sources), then you can make your sample look like your population, at least demographically. That can help ensure you don’t have an obvious demographic bias in the data. Particularly if you know, for instance, that your convenience sample over-represents a certain group (for example, if there are more women on the survey panel than you are using to draw your sample), you can limit the number of women in your sample, so the sample will look like your population.
Demographics Based on Behavior, Attitudes, etc.
That's only necessary because we know that demographics often correspond with different attitudes and behaviors. For instance, young people behave differently than older people; males and females shop and buy differently; upper income people have different attitudes than lower income people.
How Extensive Should it be?How far do you need to go with matching your sample demographics to the population? If a characteristic doesn’t impact the attitudes or behaviors that you are studying, then there’s no need to go through the extra trouble or expense to make your sample match your population on that criterion. For instance, if your study is about skincare, and if people who live in the Northeast don’t have different attitudes or behaviors around skincare compared to people in the Midwest, then no special steps will be needed to make sure your skincare sample contains a certain proportion of people from the Northeast versus the Midwest.
2. Recruitment From Multiple Sources
If the panel you are drawing your sample from recruits in a specific way (maybe just pulls people off a few specific websites, for example), then that can cause the whole panel and any sample drawn from its subject to bias in a particular direction. You should investigate how panels are recruited and built to make sure they come from a wide variety of sources.
3. Survey Incentives
Incentives also mitigate bias. Incentives exist in research not really to compensate people, but to get a broader cross-section of people to respond than otherwise would.
If you went out to people and said, "Hey, will you take my survey?" There's a group of people who will do that just because they like to give their opinion. And if you keep at it, you might get a decent sample size just from those who voluntarily agree. However, if you only take those people into your study, you're going to get a bias because the type of people who respond. Even though they enjoy surveys, they don’t always fully represent the population you are studying.
Incentives really exist for people who would think, "Yeah, I don't really have time to take a survey - I have things to do." But because you offer an incentive, they will take some time to complete your study. The incentives are really there to balance what's called non-response bias. Some people who otherwise would not respond will do so, if you give them a small incentive. It's not intended to make anybody rich or make anybody want to quit their day job and just do surveys on a permanent basis.
There is still some non-response bias because some people will not participate even with an incentive. But you do your best to minimize that.
Mitigating Bias with Field Agent
Mitigating bias might seem like a difficult task, however Field Agent makes it simple by allowing businesses to choose their own sample base with our large network of agents from all types of demographics. By doing it yourself, you'll be able to post jobs to screen Agents and mitigate any bias from your sample. Get started today.
Continuing Research Education
Learn more about the Basics of Research and Sampling Methods in our past blog posts. Be sure not to miss out on our series of eBooks that will walk you through research and how to use it for your business.