Research user pain points and opportunities for improvement in the current panelist experience
Audit and provide an analysis of the current panelist experience and deliver any opportunities for improvement in the areas of retention, accessibility, and/or reaching unique audiences
HIGH LEVEL TIMELINE
Jan - May, 2021
MAKE OF THE TEAM
UX Research lead
UX Design lead
Design concepts + Research report
We use a mixed methods research process to gather and analyze user pain points and provide panel dashboard design concepts through a research report presentation.
UX Research lead + UX Design lead
I conducted a competitive analysis of common usability testing platforms, gathered anecdotal evidence and reviewed existing literature, designed and delivered online surveys, questionnaires, semi-structured interviews, remote moderated and unmoderated user testing. In the research to design phase, I developed sketches, wireframes, and final design concepts.
UserTesting.com is a usability testing platform that provides both quantitative and qualitative feedback on almost any customer experience you can imagine including websites, prototypes, experiences and much more.
Sponsored by UserTesting.com as part of Georgia Tech's M.S. HCI Project Studio course, the main stakeholders of this project included a team of panel-experience experts at UserTesting.com and our end users, UserTesting.com panelists. There are two user groups at UserTesting - customers, who receive feedback on their product, and panelists, who review those products and provide their feedback.
The problem presented was to 'research common pain points and look for opportunities for improvement in the current panelist experience'. All research was conducted in a fully remote manner.
Integrating qualitative and quantitative data collection to understand user needs, collect rich user feedback, and deliver actionable insights to stakeholders
MIXED METHODS RESEARCH
Quantitative research methods are great for gathering hard data to measure, validate, and inform important decisions. However, while quant data can tell us the 'what', it often leave us with questions that only qualitative data can answer: the 'why'.
Using a mixed methods approach, we collect both quantitative and qualitative data throughout various steps across the research process and address multiple research objectives to understand user pain points, motivations driving user behavior, and how this can impact the end user experience as a whole.
Phase1 Pain Points
Onboarding, Customer service, Payment, Screening, UI, Accessibility
Phase2 Pain Points
Unmoderated user testing
Moderated user testing
Phase3 Pain Points
Screening, Communication, UI
OBJECTIVE - To gather information on pros and cons of each platform and what aspects of each testing platform users like and dislike
DETAILS - Review of existing literature and testimonials of common usability testing platforms including: Playback UX, TryMyUi, UserZoom, Lookback
FINDINGS - Possible pain points: payment, testing interface, the onboarding process, availability of tests, customer service, and screening questions
OBJECTIVE - To quickly gather large quantitative data that helps narrow down pain points through an unbiased perspective from users
DETAILS - Qualtrics surveys distributed on Reddit r/usertesting completed by 27 participants who met user criteria
FINDINGS - 71% of participants indicated that customer service was their worst experience
OBJECTIVE - To gather detailed qualitative data about particular changes users want to see in their experience as panelists while having the flexibility to probe participants about their responses
DETAILS - Remote interviews of 2 volunteer participants recruited from Reddit using a carefully crafted interview script with one interviewer and one notetaker present
FINDINGS - Pain points: screening and communication. For screening, users had trouble with the availability of tests, repetitive questions, and question formatting. As for communication, users expressed concerns about communication with customer support and communication with customers in terms of feedback.
UNMODERATED USER TESTING
OBJECTIVE - To gather a mixture of quant and qual data regarding the most pressing needs of users on the UserTesting platform and compare this to our findings from the Reddit research to further narrow down user pain points
DETAILS - Unmoderated, survey-style tests including a mixture of verbal, rating, multiple choice, and written response questions released on UserTesting.com to 10 participants divided into two different user groups
FINDINGS - Pain points: screening, communication, and system UI. For screening, users expressed trouble with being eligible for tests, frustration from seeing repeated screeners, and difficulty finding a direct link for 'help', particularly within tests. For communication, users recalled having negative experiences while trying to contact customer support and reported a desire to communicate with customers through receiving performance feedback. Finally, users expressed that they wanted to see improvements in system UI, particularly with the user dashboard. This included wanting to see star ratings directly on the user dashboard and improvements with test loading time.
OBJECTIVE - To transition the focus of our testing from narrowing down pain points to finding specific design changes end users want to see regarding pain points finalized through unmoderated testing
DETAILS - Moderated interview-style testing with 5 users recruited through UserTesting.com who met participant criteria. Participants were asked to share their screen so that we could see how users navigate through the dashboard and interact with the system regarding user pain points.
FINDINGS - We categorized our findings into the areas of screener, customer service, and feedback. We found that these categories pointed to problems that our users experience and specific design changes that our users want to see regarding those areas.
SYNTHESIS + ANALYSIS
Journey maps help visualize the main interaction-points of our users’ journey on UserTesting from: registration, screening, test-taking, feedback, to payment. This is a great way to visualize users’ journey, empathize with their experience, and organize the main pain points.
Affinity diagrams help organize information into groups of similar items or themes. We went through several sessions of affinity mapping throughout our research process and conducted a quick retrospective to flesh out the data based on: what’s good, what’s bad, ideas, and actions. Finally, we synthesized all of our unmoderated data into ‘areas of focus’ to help guide our research to design process.
EVOLUTION OF PAIN POINTS
P1. Onboarding, customer service, payment, screening, system UI, accessibility
P2. Screening, communication with UserTesting + customers
P3. Screening, communication with UserTesting + customers,
Following data synthesis and analysis, we received feedback from experts to narrow down design changes that our users wanted to see and created a feasibility/ needs matrix to help us visualize which changes were in the ‘optimal’ area of high need and high feasibility.
After narrowing down some possible design changes, we developed design ideas and presented them to the Director of Product Design during a co-design activity where we discussed which design ideas would be of high value to UserTesting. In the end, we decided to work on design ideas concerning customer ‘Feedback’ and the panel ‘Dashboard’
After our co-design activity transitioned into design. We created a mockup of the panel dashboard and made some changes to the upper menu bar, the drop-down menu that you see when you click on your user profile, and to the feedback options that customers see from their end.
We made some sketches and wireframes on Balsamiq to show how we would implement these changes and what they would look like before we proceeded with our redesign concepts. Finally, we implemented user feedback and made specific changed to the panel dashboard in a mockup of design concepts created on Figma.
Narrowing down a broad research prompt
Multiple stakeholders with different needs
Making informed decisions with actionable insights based on user feedback
One of the biggest learnings from this semester project was turning a really broad and ambiguous research question into specific actionable design insights that could have real impact. To do this, we triangulated our research into background research (competitive analysis, anecdotal experience as a panelist, literature review), Reddit research (survey and interview), and UserTesting research (unmoderated and moderated research on UserTesting). We also met with the panel experience team at UserTesting to discuss and compare research findings and receive feedback on our designs. This process helped us narrow down the pain points we wanted to focus on and provided us a lot of different data points we could look at to give us insight into our problem space.
Balancing the technical and business constraints of UserTesting with our users’ needs was another challenge. Often times the main pain points we uncovered were not feasible or realistic for UserTesting to actually make an impact. To manage this, we made sure to involve our stakeholders often in the research process, getting feedback on what we were finding and any areas in which we could dig a little deeper into.
Finally, the biggest learning we had was making informed decisions based on user feedback, iterating on our research ideas and design ideas with experts, and including the users as much as possible in the design process. All of these factors allowed us to focus on the user, uncover what they needed most for a better experience, and decide what design insights were most feasible.