Momentive: Key Driver Analysis
Maximize sales impact with Key Driver Analysis by Momentive.
Overview
Momentive’s previous research studies demonstrated that customers need market prediction services that can accurately predict how well a new product concept will perform with customers. Myself and another designer were tasked with the research and design of an analysis feature that leveraged Momentive’s people-powered data to predict what product and service attributes are most likely to drive market performance.
Challenge
Competitors within the Market Research space, like Nielsen, offer this type of service by analyzing real market data such as sales numbers. However, Momentive’s business model relies on survey data, which requires a different approach.
Opportunity
Build a straightforward tool to enable customers to launch new products with confidence and maximize sales impact by showing which attributes are highly correlated to purchase intent.
Results
Supported sales motion through customer-driven feature development and through rapid iteration and testing increased qualitative measures as follows:
25% improvement in ease of use
20% improvement in appearance
15% reduction in task completion time
My role
UI/UX
Competitive analysis
User research and testing
Interaction design
Copy writing
Team
Sr. Product Designer
Greg Allen
Sr. Product Designer
Caio Braga
Sr. Product Manager
Adam Di Tota
Sr. Engineering Manager
Adam Klockars
Sr. Software Engineer I
Chigozie Beluolisa
Discovery
Leveraging existing research on Spearman’s Correlation Coefficient
The CX pillar conducted a study in Q1 2020 to determine which algorithm to use for NPS Key Driver analysis, and documentation from that project was used as a reference for the current project. However, there is no central knowledge base for the organization.
To meet the aggressive goal of creating a Proof-of-Concept in Q4 2020, the decision was made to use the Spearman's algorithm path. This decision was based on positive results from tests performed by the research team, the algorithm's suitability for the organization's attribute data model, and its more straightforward implementation compared to a regression model. Although the regression model is still recommended for future iterations, it would require further investigation, and the Spearman's algorithm was considered the best option based on the investigation conducted earlier in the year.
Investigating how direct competitors and other tools address the need
The analysis found that among the three audited competitors, only Zappi supported key driver analysis explicitly. However, it used Kruskal's algorithm and could only analyze one stimulus at a time. The bar chart format used by these competitors was easy to understand but could become cumbersome for many attributes, and it was unclear how it supported multiple stimuli.
Qualtrics lacked a key driver analysis tool, but customers could create this visualization on Stats IQ using different algorithms, including Spearman’s. Qualtrics used a four-quadrant chart format that may not have been easily understood by customers outside of market research.
Alchemer did not have a dedicated key driver analysis tool, but it provided Pearson's chi-squared on their crosstabs, allowing customers to identify product attributes that may have affected the likelihood of purchase.
Data visualization considerations
A scatter plot is a valuable tool to visualize relationships between multiple variables, which is essential for Key Drivers analysis. It allows customers to compare concepts, their main goal with Concept Testing.
However, there are two main challenges when using a scatter plot:
It can be difficult to understand when there are more than two dimensions.
They can be challenging to interpret due to the density and complexity of the information presented, particularly when comparing different concepts. Moreover, correlation analysis involving more than two dimensions can further compound the complexity.
SurveyMonkey's GetFeedback uses a custom visualization to show attribute strength and rank order. However, their strength indicator is not a widely recognized visual pattern and does not suggest how to act based on attribute performance.
Design
Identifying opportunities to enhance the user experience
We started by generating a few designs, which we reviewed internally with our design team and subject matter experts (SMEs). After careful consideration, we ultimately settled on a design we felt would be well-received by our customers. To confirm this, we conducted testing with several existing concept-testing customers who aligned with our target user personas. While the feedback we received was mostly positive, we did identify some areas where we could improve the user interface and experience, including:
Displaying attribute data points from other concepts, but in a reduced opacity, resulted in an overwhelming number of data points for customers.
The label for the relative scale went unnoticed, and some customers were concerned that it skewed the distance between points. They would like the ability to see the absolute scale but were unsure how to change it.
While more advanced market researchers were familiar with the chart, both novice and experienced users would benefit from a summary view to communicate important insights and recommended actions to leadership.
Increasing speed to insight with local AI insights
In the follow-up rounds of testing, I focussed on addressing some of the feedback customers had given us. Working with the ML team, we developed a model and logic that would call out the most significant results and which insight would get displayed if the results were close.
Through testing color, language, and iconography, I devised a badge-based approach to quickly communicate to customers how they could take action on the attributes.
Reducing complexity to enable users to quickly identify insights
The previous approach to displaying all stimuli data points was overwhelming and slowed down the process of uncovering insights for customers. However, by reducing the number of data points shown in the user interface (UI), customers were able to more easily identify patterns and relationships in the data, which led to faster insights. This ultimately saved customers time and improved their overall experience.
Solution
Unlocking insights with our user-friendly and intuitive Key Driver Analysis solution
Our team has worked tirelessly to develop a solution that takes into account the needs of novice and expert market researchers alike while ensuring that the tool is intuitive and easy to use. We've taken feedback from our customers, and incorporated their input to create a tool that strikes the perfect balance between usability and advanced features.
The final solution for key driver analysis includes a sleek and modern interface that is visually appealing and easy to navigate. It is designed to be user-friendly and accessible for individuals at all levels of expertise in market research, with clear labels and instructions for every feature.
We've also improved task completion time and learnability through enhanced functionality and features that are intuitive and streamlined. As a result, our users will be able to complete their research studies more efficiently and with greater accuracy.
Finally, our solution has proven to generate increased revenue for our clients, demonstrating the powerful insights that can be gleaned from key driver analysis. We are confident that our final solution will exceed the expectations of our users and help them make data-driven decisions that drive business success.
