Market & Concept Evaluation
During the Covid-19 pandemic, it became crucial to keep track of parents’ interest in educational products as their needs for home learning resources were rising. I worked with the product team to gather research questions the team needs answer for, then collaborated with other researchers to identify key metrics (e.g., general interest in product category) that need year-over-year comparisons.
I designed a market research survey to evaluate parents’ interest in the general product category and learning subject along with product-specific questions, such as parents’ purchase interest based on a detailed product description. Then I programmed and launched the survey on SurveyMonkey with purchased target audience panel (e.g., parents who have 6-8 year old children in the United States).
- Math is the learning subject that parents of 6-8 year old are most interested in investing in.
- Potential gap in the market: Interest in purchasing an interactive math game increase significantly since March 2022, but parents who reported owning this type of educational products remained comparable.
- Parents are interested in the product concept described in the survey. The price estimates for the concept is lower than expected, indicating the need for better design and marketing strategies to convey value.
Concept and Gameplay Validation
At the stage of Alpha, we need to validate the product concept with families who meet the target audience criteria for this product. Our goal is to set the baseline metrics (e.g., Net Promoter Score, Difficulty of Gameplay) for this prototype while inviting testers to share their inputs in open-ended responses.
When the product team prepared to launch the prototype on TestFlight, I started recruiting and screening 60~80 testers from our internal pool of testing families. I designed a short survey to confirm testers’ eligibility and availability, and coordinated with our scheduler to process testers’ NDA and consent forms.
To prioritize the questions in the Alpha survey, I met the cross-functional stakeholders to discuss and track changes maded to the survey draft. My goal is to ensure the key questions from product managers, designers and engineers are translated into a suitable format. When I completed the survey draft, I incorporated feedback from stakeholders and other researchers to revise and rebalance the draft.
Some of the key insights from the Alpha survey synthesis: (see the sample slide below)
- The educational content was challenging to the 6 year olds who struggled with mental math.
- Children had a hard time distinguishing the physical manipulatives.
- Some pieces (e.g., balloon of “1”) were too small for children’s fingers.
- The core gameloop of building an airship to deliver packages felt repetitive as children had limited control with the airship and constrained interactions with the game characters.
At the end of the sysnthesis, I presented the findings and recommendations to corresponding stakeholders. Based on our discussion, we decided to move forward to the Beta Evaluation in a few months to compare the results between Alpha and Beta.