Whatsapp diary study

Yup, you read that right I used WhatsApp messenger to conduct a diary study. Yup, it felt like a full-time job chatting with 20 participants for 7 days (including the weekend😲), but I would 100% do it again.

“A diary study is a research method used to collect qualitative data about user behaviors, activities, and experiences over time. In a diary study, data is self-reported by participants longitudinally — that is, over an extended period of time that can range from a few days to even a month or longer. During the defined reporting period, study participants are asked to keep a diary and log specific information about activities being studied.”
NNgroup
whatsapp-promo.png

WhatsApp & our users

WhatsApp was chosen as through a marketing audit of the Boiler Room community, the majority of users were familiar and confident with the instant messaging app. Weighing this up against onboarding them with more specific tools would have made the study longer and potentially cause participants to opt out prior to the start.

WhatsApp & budget

Money is a sensitive topic, but with WhatsApp being a free app, It meant the research budget which was partially going to be used to pay for a specific diary study tool, was used to reward participants for their time and effort over the 7 days.

WhatsApp & Drop off rates

Although every diary study goes through the onboarding phase of recruiting committed participants, things happen and sometimes you don't get a reply. The joy of using WhatsApp meant I was able to quickly see who had read messages and prompt them in a friendly manner. Messages sent were always in a relatable Boiler Room-esque tone of voice so that users never felt as if they participating a repetitive study. What was interesting and refreshing during the study was some users became proactive and treated the messages as a personal diary without any prompts, which went beyond what I expected and gave the results more context.

WhatsApp diary study in action

Learnings

  • Only use WhatsApp if it's an app your user base are familiar with.

  • Download the WhatsApp app on a laptop, desktop or tablet, the larger screen made managing participants messages and off script personal replies easier as well as giving a fuller view of the participants as a whole.

  • Plan your messages thoroughly and maintain a tone of voice, I scripted every message I could send so that depending on participant replies/ lack of replies I didn't have to overthink what i had to say.

  • Due to the speed of replies, data collection can be a time consuming task if left until the end. I’d recommend making an excel sheet split by participants, and inputting their replies as they come in or end of day.
     

I’ve attached my Listening Habits Test Plan for anyone who wants to give this a try.

User research when everyone loves your brand!

User research at Boiler Room is none like anywhere else. Through my experience at agencies and client side, we always had to include a hefty monetary incentive to entice users and even then I had to cross my fingers that they would show up.

This was not the case at Boiler Room, with 1.9 million Facebook followers and a private community of 9000 fans at my disposal, my biggest issue was our incentives were too good and all our fans wanted exclusive show entry and merchandise.

The three biggest issues I faced were:

  • Recruiting the right users

  • Overwhelmed and nervous feedback

  • When to present the incentive

Inspiration from Google

Through leading the Boiler Room community and holding regular user research sessions I became familiar with the fans. Some fans simply just wanted to visit our HQ and get the incentives without participating in any user research session.  Although we had a screener in place I had to be diligent in recruiting the right user as time was always precious.

Taking a leaf out of Google’s Captcha.

I introduced another screener typically in the form of a video call, voice call or another survey as a way to separate useful users from our fans to ensure the right users came in.

Every little thing is gonna be alright

At Boiler Room, things move at 100mph, from teams travelling across the globe to your favourite artist or Dj popping in. I always felt user's apprehensiveness prior to entering our HQ.

To limit this and make them feel comfortable:

  • I always met users at the door and greeted them by name

  • Dedicated 10 minutes to showing users around our HQ and to key rooms they’d be aware of from our Live shows

  • Introduced users to members of the Boiler Room team and anybody they knew (Users always knew someone).

The Incentive

Through past experiences, users were given their incentives at the end of user research sessions. However, with our offering of exclusive show entry and merchandise, waiting until the end didn’t work as users were eager to say the right thing to guarantee they’d receive the incentive. I thought it was simple, hand over the incentive early and everything is all good… I was wrong! Offering the incentive early sparked up a conversation that compromised the duration of the session. Through mixing and matching a tactic rose to the top:

Let the incentive be seen- During the 10 minute show around, I informed users that their incentive was waiting for them. Their incentives were placed in the meeting room where the session was being held with their names clearly visible. These simple tactic helped them feel at ease and allowed the session to flow.