top of page

Research user pain points and opportunities for improvement in the current panelist experience



Audit and provide an analysis of the current panelist experience and deliver any opportunities for improvement in the areas of retention, accessibility, and/or reaching unique audiences


Jan - May, 2021
(15 weeks)


Jason Dong

Stephanie Lee


UX Research lead

UX Design lead


Design concepts + Research report

We use a mixed methods research process to gather and analyze user pain points and provide panel dashboard design concepts through a research report presentation.


UX Research lead + UX Design lead

I conducted a competitive analysis of common usability testing platforms, gathered anecdotal evidence and reviewed existing literature, designed and delivered online surveys, questionnaires, semi-structured interviews, remote moderated and unmoderated user testing. In the research to design phase, I developed sketches, wireframes, and final design concepts.

BACKGROUND is a usability testing platform that provides both quantitative and qualitative feedback on almost any customer experience you can imagine including websites, prototypes, experiences and much more. is a usability testing platform that provides both quantitative and qualitative feedback on almost any customer experience you can imagine including websites, prototypes, experiences and much more.


Sponsored by as part of Georgia Tech's M.S. HCI Project Studio course, the main stakeholders of this project included a team of panel-experience experts at and our end users, panelists. There are two user groups at UserTesting - customers, who receive feedback on their product, and panelists, who review those products and provide their feedback.


The problem presented was to 'research common pain points and look for opportunities for improvement in the current panelist experience'. All research was conducted in a fully remote manner.


GTxUT - Frame 1.jpg

Mixed Methods

Integrating qualitative and quantitative data collection to understand user needs, collect rich user feedback, and deliver actionable insights to stakeholders


Quantitative research methods are great for gathering hard data to measure, validate, and inform important decisions. However, while quant data can tell us the 'what', it often leave us with questions that only qualitative data can answer: the 'why'.

Using a mixed methods approach, we collect both quantitative and qualitative data throughout various steps across the research process and address multiple research objectives to understand user pain points, motivations driving user behavior, and how this can impact the end user experience as a whole.

Background Research

Competitive analysis

Anecdotal evidence 

Phase1 Pain Points

Onboarding, Customer service, Payment, Screening, UI, Accessibility 

Reddit Research


Semi-structured interviews

Phase2 Pain Points

Screening, Communication

UserTesting Research

Unmoderated user testing

Moderated user testing

Phase3 Pain Points

Screening, Communication, UI


OBJECTIVE - To gather information on pros and cons of each platform and what aspects of each testing platform users like and dislike 


DETAILS - Review of existing literature and testimonials of common usability testing platforms including: Playback UX, TryMyUi, UserZoom, Lookback

FINDINGS - Possible pain points: payment, testing interface, the onboarding process, availability of tests, customer service, and screening questions


OBJECTIVE - To quickly gather large quantitative data that helps narrow down pain points through an unbiased perspective from users


DETAILS - Qualtrics surveys distributed on Reddit r/usertesting completed by 27 participants who met user criteria

FINDINGS - 71% of participants indicated that customer service was their worst experience


OBJECTIVE - To gather detailed qualitative data about particular changes users want to see in their experience as panelists while having the flexibility to probe participants about their responses


DETAILS - Remote interviews of 2 volunteer participants recruited from Reddit using a carefully crafted interview script with one interviewer and one notetaker present

FINDINGS - Pain points: screening and communication. For screening, users had trouble with the availability of tests, repetitive questions, and question formatting. As for communication, users expressed concerns about communication with customer support and communication with customers in terms of feedback.


OBJECTIVE - To gather a mixture of quant and qual data regarding the most pressing needs of users on the UserTesting platform and compare this to our findings from the Reddit research to further narrow down user pain points


DETAILS - Unmoderated, survey-style tests including a mixture of verbal, rating, multiple choice, and written response questions released on to 10 participants divided into two different user groups

FINDINGS - Pain points: screening, communication, and system UI. For screening, users expressed trouble with being eligible for tests, frustration from seeing repeated screeners, and difficulty finding a direct link for 'help', particularly within tests. For communication, users recalled having negative experiences while trying to contact customer support and reported a desire to communicate with customers through receiving performance feedback. Finally, users expressed that they wanted to see improvements in system UI, particularly with the user dashboard. This included wanting to see star ratings directly on the user dashboard and improvements with test loading time.


OBJECTIVE - To transition the focus of our testing from narrowing down pain points to finding specific design changes end users want to see regarding pain points finalized through unmoderated testing


DETAILS - Moderated interview-style testing with 5 users recruited through who met participant criteria. Participants were asked to share their screen so that we could see how users navigate through the dashboard and interact with the system regarding user pain points.

FINDINGS - We categorized our findings into the areas of screener, customer service, and feedback. We found that these categories pointed to problems that our users experience and specific design changes that our users want to see regarding those areas. 



Journey maps help visualize the main interaction-points of our users’ journey on UserTesting from: registration, screening, test-taking, feedback, to payment. This is a great way to visualize users’ journey, empathize with their experience, and organize the main pain points.

Group 1.jpg


Affinity diagrams help organize information into groups of similar items or themes. We went through several sessions of affinity mapping throughout our research process and conducted a quick retrospective to flesh out the data based on: what’s good, what’s bad, ideas, and actions. Finally, we synthesized all of our unmoderated data into ‘areas of focus’ to help guide our research to design process.

GTxUT - Focus_ Moderated test.jpg




P1. Onboarding, customer service, payment, screening, system UI,         accessibility

P2. Screening, communication with UserTesting + customers

P3. Screening, communication with UserTesting + customers,     

     system UI

Screen Shot 2021-06-18 at 5.32.43 AM.png



Following data synthesis and analysis, we received feedback from experts to narrow down design changes that our users wanted to see and created a feasibility/ needs matrix to help us visualize which changes were in the ‘optimal’ area of high need and high feasibility.


After narrowing down some possible design changes, we developed design ideas and presented them to the Director of Product Design during a co-design activity where we discussed which design ideas would be of high value to UserTesting. In the end, we decided to work on design ideas concerning customer ‘Feedback’ and the panel ‘Dashboard’ 


After our co-design activity transitioned into design. We created a mockup of the panel dashboard and made some changes to the upper menu bar, the drop-down menu that you see when you click on your user profile, and to the feedback options that customers see from their end.


We made some sketches and wireframes on Balsamiq to show how we would implement these changes and what they would look like before we proceeded with our redesign concepts. Finally, we implemented user feedback and made specific changed to the panel dashboard in a mockup of design concepts created on Figma. 


  • Narrowing down a broad research prompt

  • Multiple stakeholders with different needs

  • Making informed decisions with actionable insights based on user feedback

One of the biggest learnings from this semester project was to narrow down an ambiguous research question into specific actionable design insights. To do this, we triangulated our research into background research (competitive analysis, anecdotal experience as a panelist, literature review), Reddit research (survey and interview), and UserTesting research (unmoderated and moderated research on UserTesting). We also met with the panel experience team at UserTesting to discuss and compare research findings and receive feedback on our designs. This process helped us narrow down the pain points we wanted to focus on and provided us a lot of different data points we could look at to give us insight into our problem space.


Balancing the technical and business constraints of UserTesting with our users’ needs was another challenge. Often times the main pain points we uncovered were not feasible or realistic for UserTesting to actually make an impact. To manage this, we made sure to involve our stakeholders often in the research process, getting feedback on what we were finding and any areas in which we could dig a little deeper into.


Finally, the biggest learning was making informed decisions based on user feedback, iterating on our research ideas and design ideas with experts, and including users as much as possible in the design process. All of these factors allowed us to focus on the user, uncover what they needed most for a better experience, and decide what design insights were most feasible.

let's chat

  • LinkedIn
bottom of page