Sampling is a term that comes up a lot when working with data. You might have someone ask you “How big is the sample size?” for a study. Or you might come across terms like “random sample” or be warned against, “double-counting.”
But, what exactly is a sample?
A sample is a subset of the population being studied. This subset was selected by the evaluator, and data will only be collected from that subset. For example, if 100 visitors come to a museum in one day and 10 of them are invited to fill in a survey, those 10 visitors are the sample. This means that the majority of visitors are not included in the sample.
So does this mean the museum doesn’t care about the 90 visitors who were not invited to fill in a survey? No – the museum is just trying to use their resources as efficiently as possible.
Sampling is necessary because it is usually impossible (and unnecessary) to collect data from every single visitor. But sampling isn’t just about picking a random 10% of visitors to study – even in simple visitor studies.
Here are a few things to consider when selecting a sample for a simple visitor study:
- Are you collecting quantitative (numbers) or qualitative (words) data? You’ll need a larger sample size for quantitative data compared to qualitative data.
This is because quantitative data is more likely to accurately represent the population being studied when you have a larger sample size. For example, if you want to know the average age of visitors at an interpretive centre you’ll get a more accurate response if you ask 100 visitors, compared to only 10.
Qualitative data is great for understanding visitor experiences, values and motivations. It is used to explore topics more deeply (instead of how old a visitor is, you would ask them how they feel about their age). It’s also generally more resource intensive compared to collecting and analysing quantitative data. This makes qualitative data great for exploring topics and highlighting individual experiences which means a smaller sample size can be more appropriate.
- When are you collecting data from your sample? If your goal is to observe how visitors behave in an exhibit, it’s important to make sure you collect data on different days and at different times of day.
This is especially important if you are doing an evaluation at a site that is affected by the weather or other seasonal factors. So, if you are the one collecting data, this means you’ll need to plan to do some work on weekends and holidays.
- Who are you targeting in your sample? And consider who you are not targeting.
If you are interested in what motivates families to visit your site, have a clear definition for “family.” Does it include grandparents with grandchildren? Does it include large extended families? Would you like to include families with older teens or are you only interested in families with younger children?
A good way to figure out who you want to target for your sample is to think of who you do not want to include. And think back to the goal of your visitor study – what do you want to know?
One more thing to consider is how to avoid double counting. If you are conducting a study at a site with high repeat visitation – how will you make sure you don’t accidentally skew your results by counting the same people more than once?
- How will you minimize bias? We are all biased (yes, you and your visitors).
The safest way to minimize bias in your sample is to pick a random method of sampling. For example, invite every fifth visitor through the front door to fill in a survey or if you are timing and tracking, track the next visitor who enters the exhibit once you finish the previous tracking sheet.
Just don’t pick something too specific, like only sampling visitors who wear corduroy pants or purple hats because that would bias your sample to visitors who wear corduroy pants and purple hats…
This one can be challenging. As you collect data, monitor yourself for any bias – it can creep in in unexpected ways!
- What is your rate of non-response? Think of how you will record it and what it might mean.
Not every visitor will want to fill in a survey and that’s ok. Part of thinking about sampling is considering your rate of non-response. If you invite 250 people to fill in a survey and only 10 visitors complete it, it is worth considering if there is something about the survey that is making it inaccessible to the remaining 240 visitors.
Different data collection methods will have different response rates, and it is very normal to have a low response rate. But if your response rate is very low, this means you might be missing a critical part of your sample, which will skew your results.
So, what’s the magic number for sampling? As you have have probably already guessed that it depends on the resources you have to complete the study and the methods you use.
TM Visitor Studies 