Surveys are a staple in visitor studies. They are this – almost magical – tool you can use to ask deep questions that help you understand visitor’s motivations, experiences, values in addition to collecting demographics. They can be long or short. Online or in person. You can translate them into different languages and collect data from visitors you would never be able to speak with otherwise.
The flexibility and (perceived) simplicity of surveys makes them a popular choice in evaluation. However, this is also where you can run into trouble. There are so many questions you can ask with a survey and this can feel overwhelming. It is also easy to include questions you don’t actually need to ask. Designing a survey that will actually get enough usable responses, that are useful to you, is hard.
I recently did a review of surveys I’ve run in the past couple of years to look for what worked well and what didn’t.
I found successes but also…areas for growth…in three categories:
- Phrasing
- Format
- Delivery
This blog post is the first part of a series where I will break down mistakes from all three categories.
Phrasing
Clarity is critical in surveys. Even if you’re delivering them in person, you aren’t going to be reading them over with each respondent to check for comprehension.
I once asked: “How many people are in your group today?”
A simple question, right? Wrong. Respondents had no idea whether to count themselves or not. I’m sure many made an assumption one way or the other but it’s not clear in the question. This meant I ended up with a bunch of responses that were either accurate, or off by one.
Now, what about this question: “Where do you live?”
Asking visitors where they live can feel awkward but it can also be useful to know. I’ve tried asking this a few ways. I almost always make it a closed response question so respondents can pick from a list of municipalities, regions and/or write in a response under “Other.”
Here are a couple of ways I’ve asked this question:
“Where are you visiting from?”
Too vague. Some respondents thought I was referring to where they were visiting from that day and listed where they were staying in a hotel or Airbnb. This could be useful information but it’s not what I was trying to get with this question.
“Where is your permanent residence?”
Pretty formal. While this question could feel a little invasive in its directness, the main issue I’ve found is that some respondents are living somewhere that is different from their permanent residence. Maybe they’re students, travelling or in the process of a big move. The answer to this question tells you what’s on their ID but it might not tell you where they are actually living.
There is so much you could ask about where visitors are coming from. If you are struggling to phrase a question like this, ask yourself what you really want to know.
Do you want to know how many visitors are considered local because you need the stats for funding purposes? Maybe you want to know how long most visitors are willing to commute to your site. Or, perhaps you want to dig into census data and need your visitors’ postal codes to do that. Once you know what information you need, it will be easier to ask the right question.
Making mistakes – and learning from them – is part of building new skills. I’m happy to share some of my survey slipups so you don’t have to experience them firsthand.
Just remember, each site and survey is unique so chances are you’ll still get to make some mistakes of your own. Plan for this by building in extra time to test your surveys, and to make changes based on those tests. If you catch an issue after you’ve finished collecting data this doesn’t mean you are done for – just be transparent about how it will affect your results and make note of what you can change in the future.
Stay tuned for Part 2.
Learn more about surveys in these beginner friendly courses designed for museum and heritage professionals:
TM Visitor Studies 