Reforge · AI Interviewer
Run interviews in your sleep
How I led a team to build an AI interviewer, concept testing and prototype testing in nine months.
In 2025, Reforge set out to help product teams hear from their customers more often, without adding hours to their week. Over nine months, our team built an AI interviewer that holds live voice conversations with participants, then extended it into concept and prototype testing so teams could test ideas before committing engineering time.
By April, teams had created 880 interviews with a 4.04 out of 5 participant rating. Customers said the live voice format set it apart, and one grew from a 200-minute plan to 1,500 minutes, calling it the one workflow they couldn’t replace. It’s one of the projects I’m most proud of, a feeling shared by everyone who worked on it.
Product teams skip talking to customers not because they don’t want to do research or test their features, but because scheduling, no-shows, translation and analysis are slow and costly. To address this, we set out to develop a reliable, trustworthy and engaging AI Interviewer to help teams scale their research.
As we spoke to customers, we learned that teams also wanted more efficient ways to test concepts and prototypes before committing engineering time, especially at the pace they were building.
I served as the Product Lead on the AI Interviewer, concept testing and prototype testing, working with:
- Shane McDermed, Engineering
- Stewart Eaton, Engineering
- Lamar Oliphant, Engineering
- Tim Kao, AI Engineering
- Cole Hoffer, Engineering
- Jeffrey Chupp, Engineering Manager
- Jimmy Daly, Marketing
- Archie Leeming, Lead Product Designer
- David Kutas, Design Engineering
Archie built the design foundation for the AI Interviewer. I collaborated with him before taking over design for the next phase of the Interviewer build, concept testing and prototype testing. David built the research-page UI overhaul and connected our design system to prototype generation. Cole contributed to the interviewer’s architecture and analysis.
- Jun 2025 Set the AI Interviewer MVP direction: goal and audience input, an AI-generated interview structure, five-minute conversations.
- Oct 2025 Launched Reforge Research, combining the AI Interviewer and surveys.
- Dec 2025 Defined concept testing as a template on the AI Interviewer.
- Jan 2026 Full launch of concept testing.
- Feb 2026 Launched prototype testing for Reforge Build prototypes only while we tested (Build was later folded into Miro prototyping).
- Mar 2026 Expanded prototype testing to Lovable, Figma, and v0, the prototyping tools our customers used most.
The AI Interviewer runs live voice interviews, then two templates built on it let teams test concepts and prototypes.
Setting up
The participant experience
1. Welcome. Participants open the interview link, enter their email and choose an interview language. Teams can fully customize this screen.
2. Microphone access. A clear prompt explains why the interview needs the microphone and that recordings stay private.
3. The interview. The AI interviewer dynamically follows up with each participant based on how they respond and interact with the prototype.
4. Thank you and rating. Participants rate the interview and can leave feedback, which is how we monitored quality. Teams can customize this screen too.
Analysis
Launch videos for concept testing and prototype testing.
What customers told us about why they used it, how, and what they needed next.
Why they chose it
Live voice interviews, not a chatbot or a form. No scheduling or no-shows, no account needed for participants, and structured summaries tied to exact quotes.
How they used it
Feature sentiment and concept reactions, finding blockers for users who never converted, and internal pulse surveys. Results fed straight into backlogs and roadmaps.
The tradeoff they accepted
Answers were shallower than a human-led interview, but the volume, speed and time saved won out.
“100% worth it.”
What they wanted next
Results sent automatically to Slack, Notion and Linear; editing a live interview without breaking its link; and uploading product docs as context for sharper questions.
- Prototype tests took off fast: 106 in about eight weeks after rolling out in March.
- Each type did a different job: conversations for discovery, concept tests for reactions (95% probed concept reaction), prototype tests for task and flow validation.
- Multilingual by default: 862 of 880 interviews ran in more than one language, though most were written in English.
- Ship end-to-end, then polish. Choosing a working product over polish in December made the January launch, and its spike, possible.
- Setup is the product. Defaults stuck about 95% of the time, three-quarters of interviews never left draft, and about 3% of signups created a test. Activation was the biggest opportunity, which matches customers asking to upload product docs as context.
- Separate the foundation from the features. Combining the voice-agent refactor with concept features caused scope creep, and manual voice QA was slow.
- A full launch means owning trust. We launched fully despite occasional inappropriate interviewer responses: the risk was low and most feedback was positive. The team QA’d the interviewer extensively at our team offsite and in the several weeks before launch, and we added post-interview quality ratings and open-text feedback so we could spot and learn from any issues participants hit. When reward farming appeared, unique per-participant links fixed it within a week.
- Voice was the differentiator. Customers chose live voice over chatbots and forms, even with shallower answers than a human interview.
Recent studies back the core bet behind the AI Interviewer, with real tradeoffs.
- More depth, little cost to satisfaction. In a randomized experiment with 2,243 US panel respondents, AI follow-up questions more than doubled answer length compared with a fixed follow-up prompt, and more people gave a distinct reason (24% vs. 14%). Satisfaction barely moved, and respondents who commented were positive about the AI interviewer about six times as often as negative. (Austin et al., Public Opinion Quarterly, 2026)
- Depth at a cost. A randomized experiment with 3,160 respondents found AI-moderated interviews produced about 4.8 times as many words as written open-ends, but completion fell from 99.4% to 40.5%, mostly among people less comfortable with AI. The authors caution against treating AI interviews as a drop-in replacement for surveys. (Verasight, 2026)
Our results lined up. Participants rated interviews 4.04 out of 5, and customers accepted less depth than a human interview in exchange for the time saved. Drop-off was our challenge too, on the setup side: about three-quarters of interviews never left draft, so activation mattered as much as interview quality.
Following the Miro acquisition, we continued to support existing Reforge Research customers, with less new product development. As we formed a new team at Miro focused on concept validation, we collected customer feedback on the AI Interviewer to inform our roadmap.