Asking questions, analysing answers

You are reading contentfolks—a fortnightly blend of sticky notes, big content ideas, and small practical examples. Thank you for being here! ~fio

Hello 👋

Six years ago this month, I took a detour from content and got a job in usability research. I’m still using a lot of what I learned, especially when it comes to asking good questions and analysing answers.

I’m about to share a few tips about both—but first, a photo of yours truly in a research lab, circa 2016:

…remember in-person meetings without masks or social distancing?!?

#1: default to open-ended questions

If you’re running a survey or interviewing a customer AND you are a shy researcher / are just getting started / need to feel in control, you might gravitate towards closed-ended questions that are narrow in focus and allow limited responses (e.g. Do you like our blog? → Yes/No). That’s because closed-ended questions are safe, keep you in charge, and lead to no surprises—but they generally lead to no useful discoveries, either.

Resist the temptation to use closed-ended question and reframe them so they become open-ended instead. The process usually involves re-starting the question with a ‘what’ or a ‘why’. Examples:

Closed-ended: Did you find this article useful? → Open-ended: What did you find most/least useful about this article?
Closed-ended: Would you recommend our blog? → Open-ended: What do you like about our blog?
Closed-ended: Is your content helping you do [x]? → Open-ended: Why do you read our content?

💡 A practical example 💡

Here are a few more open-ended questions I used in recent surveys & calls.
Re-use them anytime you need to:

What is your biggest content problem to solve in the next 3 months?
Which topic would, if I wrote about it, make you think “FINALLY”?
What’s your advice on how we should improve [x]?
What’s ONE thing I should do with this newsletter from now on?
…and what’s one thing I should definitely NOT do?
How did [x] meet your expectations?

#2: get quantitative insight from open-ended questions

We think of open-ended questions as delivering qualitative insight and closed-ended questions as giving quantitative data. That’s because answers to Did you like this piece? Yes/No can easily be turned into stats and percentage points, unlike what happens when you ask What do you like/dislike about this blog?

…but you can always get quantitative information from open-ended questions, provided you know how to analyse the dataset. Though there may be tools out there that automate the process, I recommend doing it manually: it gets you really close to the data and, by extension, your audience.

💡 A practical example 💡

Here is the method I used to quantify the open responses to a contentfolks survey I ran last month:

1. Open a spreadsheet and start a tab for the first question

2. Paste all responses into column A, one response per row, starting from A2

3. Identify the main theme of the answer in row A2 and write it as heading in column B1

4. Add a 1 to cell B2 to indicate that the answer (row) matches the theme (column)

5. Repeat steps 3 and 4 for each row

⚠️ Important: when adding a new theme as you go through the responses, check previous answers that might fit in this new category and add a ‘1’ accordingly.

6. At the end of the process, use the SUM function to add up the 1s in each column:

Et voilà!, you can now offer a precise breakdown and calculate each theme as a percentage of the total. For example, after going through the process I made this bar chart that summarises my readers’ top 5 content marketing issues:

I hope this write-up come in handy the next time you run a survey, talk to your customers, and/or find yourself with pages upon pages of interview transcripts you need to make sense of.

And if you have questions at any point, email me—bonus points if you ask me an open-ended question 😉