UserTesting.com

Research user pain points and opportunities for improvement in the current panelist experience

OVERVIEW

THE PROBLEM

Audit and provide an analysis of the current panelist experience and deliver any opportunities for improvement in the areas of retention, accessibility, and/or reaching unique audiences

HIGH LEVEL TIMELINE

Jan - May, 2021
(15 weeks)

MAKE OF THE TEAM

Jason Dong

Stephanie Lee

MY ROLE

UX Research lead

UX Design lead

THE SOLUTION

Design concepts + Research report

We use a mixed methods research process to gather and analyze user pain points and provide panel dashboard design concepts through a research report presentation.

CONTRIBUTIONS

UX Research lead + UX Design lead

I conducted a competitive analysis of common usability testing platforms, gathered anecdotal evidence and reviewed existing literature, designed and delivered online surveys, questionnaires, semi-structured interviews, remote moderated and unmoderated user testing. In the research to design phase, I developed sketches, wireframes, and final design concepts.

BACKGROUND

UserTesting.com is a usability testing platform that provides both quantitative and qualitative feedback on almost any customer experience you can imagine including websites, prototypes, experiences and much more.

 

Sponsored by UserTesting.com as part of Georgia Tech's M.S. HCI Project Studio course, the main stakeholders of this project included a team of panel-experience experts at UserTesting.com and our end users, UserTesting.com panelists. There are two user groups at UserTesting - customers, who receive feedback on their product, and panelists, who review those products and provide their feedback.

 

The problem presented was to 'research common pain points and look for opportunities for improvement in the current panelist experience'. All research was conducted in a fully remote manner.

RESEARCH

GTxUT - Frame 1.jpg

Mixed Methods

Integrating qualitative and quantitative data collection to understand user needs, collect rich user feedback, and deliver actionable insights to stakeholders

MIXED METHODS RESEARCH

Quantitative research methods are great for gathering hard data to measure, validate, and inform important decisions. However, while quant data can tell us the 'what', it often leave us with questions that only qualitative data can answer: the 'why'.

Using a mixed methods approach, we collect both quantitative and qualitative data throughout various steps across the research process and address multiple research objectives to understand user pain points, motivations driving user behavior, and how this can impact the end user experience as a whole.

Background Research

Competitive analysis

Anecdotal evidence 

Phase1 Pain Points

Onboarding, Customer service, Payment, Screening, UI, Accessibility 

Reddit Research

Surveys

Semi-structured interviews

Phase2 Pain Points

Screening, Communication

let's chat

UserTesting Research

Unmoderated user testing

Moderated user testing

Phase3 Pain Points

Screening, Communication, UI

COMPETITIVE ANALYSIS

OBJECTIVE - To gather information on pros and cons of each platform and what aspects of each testing platform users like and dislike 

 

DETAILS - Review of existing literature and testimonials of common usability testing platforms including: Playback UX, TryMyUi, UserZoom, Lookback

FINDINGS - Possible pain points: payment, testing interface, the onboarding process, availability of tests, customer service, and screening questions

SURVEYS

OBJECTIVE - To quickly gather large quantitative data that helps narrow down pain points through an unbiased perspective from users

 

DETAILS - Qualtrics surveys distributed on Reddit r/usertesting completed by 27 participants who met user criteria

FINDINGS - 71% of participants indicated that customer service was their worst experience

SEMI-STRUCTURED INTERVIEWS

OBJECTIVE - To gather detailed qualitative data about particular changes users want to see in their experience as panelists while having the flexibility to probe participants about their responses

 

DETAILS - Remote interviews of 2 volunteer participants recruited from Reddit using a carefully crafted interview script with one interviewer and one notetaker present

FINDINGS - Pain points: screening and communication. For screening, users had trouble with the availability of tests, repetitive questions, and question formatting. As for communication, users expressed concerns about communication with customer support and communication with customers in terms of feedback.

UNMODERATED USER TESTING

OBJECTIVE - To gather a mixture of quant and qual data regarding the most pressing needs of users on the UserTesting platform and compare this to our findings from the Reddit research to further narrow down user pain points

 

DETAILS - Unmoderated, survey-style tests including a mixture of verbal, rating, multiple choice, and written response questions released on UserTesting.com to 10 participants divided into two different user groups

FINDINGS - Pain points: screening, communication, and system UI. For screening, users expressed trouble with being eligible for tests, frustration from seeing repeated screeners, and difficulty finding a direct link for 'help', particularly within tests. For communication, users recalled having negative experiences while trying to contact customer support and reported a desire to communicate with customers through receiving performance feedback. Finally, users expressed that they wanted to see improvements in system UI, particularly with the user dashboard. This included wanting to see star ratings directly on the user dashboard and improvements with test loading time.

MODERATED
USER TESTING

OBJECTIVE - To transition the focus of our testing from narrowing down pain points to finding specific design changes end users want to see regarding pain points finalized through unmoderated testing

 

DETAILS - Moderated interview-style testing with 5 users recruited through UserTesting.com who met participant criteria. Participants were asked to share their screen so that we could see how users navigate through the dashboard and interact with the system regarding user pain points.

FINDINGS - We categorized our findings into the areas of screener, customer service, and feedback. We found that these categories pointed to problems that our users experience and specific design changes that our users want to see regarding those areas. 

SYNTHESIS + ANALYSIS

JOURNEY
MAPPING

Journey maps help visualize the main interaction-points of our users’ journey on UserTesting from: registration, screening, test-taking, feedback, to payment. This is a great way to visualize users’ journey, empathize with their experience, and organize the main pain points.

Group 1.jpg

AFFINITY DIAGRAMMING

Affinity diagrams help organize information into groups of similar items or themes. We went through several sessions of affinity mapping throughout our research process and conducted a quick retrospective to flesh out the data based on: what’s good, what’s bad, ideas, and actions. Finally, we synthesized all of our unmoderated data into ‘areas of focus’ to help guide our research to design process.

GTxUT - Focus_ Moderated test.jpg

MAJOR FINDINGS

PAIN POINTS

EVOLUTION OF PAIN POINTS

P1. Onboarding, customer service, payment, screening, system UI,         accessibility

P2. Screening, communication with UserTesting + customers

P3. Screening, communication with UserTesting + customers,     

     system UI

Screen Shot 2021-06-18 at 5.32.43 AM.png

DESIGN

FEASIBILITY/ 
NEEDS MATRIX

Following data synthesis and analysis, we received feedback from experts to narrow down design changes that our users wanted to see and created a feasibility/ needs matrix to help us visualize which changes were in the ‘optimal’ area of high need and high feasibility.

CO-DESIGN ACTIVITY

After narrowing down some possible design changes, we developed design ideas and presented them to the Director of Product Design during a co-design activity where we discussed which design ideas would be of high value to UserTesting. In the end, we decided to work on design ideas concerning customer ‘Feedback’ and the panel ‘Dashboard’ 

DESIGN

After our co-design activity transitioned into design. We created a mockup of the panel dashboard and made some changes to the upper menu bar, the drop-down menu that you see when you click on your user profile, and to the feedback options that customers see from their end.

 

We made some sketches and wireframes on Balsamiq to show how we would implement these changes and what they would look like before we proceeded with our redesign concepts. Finally, we implemented user feedback and made specific changed to the panel dashboard in a mockup of design concepts created on Figma. 

LESSONS LEARNED

  • Narrowing down a broad research prompt

  • Multiple stakeholders with different needs

  • Making informed decisions with actionable insights based on user feedback

One of the biggest learnings from this semester project was turning a really broad and ambiguous research question into specific actionable design insights that could have real impact. To do this, we triangulated our research into background research (competitive analysis, anecdotal experience as a panelist, literature review), Reddit research (survey and interview), and UserTesting research (unmoderated and moderated research on UserTesting). We also met with the panel experience team at UserTesting to discuss and compare research findings and receive feedback on our designs. This process helped us narrow down the pain points we wanted to focus on and provided us a lot of different data points we could look at to give us insight into our problem space.

 

Balancing the technical and business constraints of UserTesting with our users’ needs was another challenge. Often times the main pain points we uncovered were not feasible or realistic for UserTesting to actually make an impact. To manage this, we made sure to involve our stakeholders often in the research process, getting feedback on what we were finding and any areas in which we could dig a little deeper into.

 

Finally, the biggest learning we had was making informed decisions based on user feedback, iterating on our research ideas and design ideas with experts, and including the users as much as possible in the design process. All of these factors allowed us to focus on the user, uncover what they needed most for a better experience, and decide what design insights were most feasible.

let's chat

  • LinkedIn
  • Instagram