Credit Karma
50 self-serve studies a quarter, and 20% less duplicate research
At Intuit Credit Karma I built the systems that let research scale without operations hovering over it: a tracked UserTesting program running about 50 studies a quarter, a company-wide research library, and the company's first in-person research program.
- Role
- Senior User Research Operations Program Manager
- Team
- Product Research and Strategy
- Timeframe
- 2022 – 2023
- Partners
- Research, design, marketing, legal, building security
- studies per quarter on the self-serve UserTesting program
- ~50
- studies per quarter on the self-serve UserTesting program
- faster recruitment
- ~40%
- faster recruitment
- faster study turnaround
- ~30–35%
- faster study turnaround
- immediate drop in duplicate research after the library launched
- 20%+
- immediate drop in duplicate research after the library launched
Overview
Credit Karma's research team had matured to the point where its work needed to be tracked and archived properly. Researchers also advised design and marketing projects that produced insights and decisions of their own, and those needed tracking too.
I took over research operations day to day and built three programs side by side: the UserTesting program, the Product Research Library and in-person research.
The challenge
Research headcount was limited and mostly embedded in verticals, while many teams without research support still needed answers to make product decisions.
Projects, artifacts and outcomes weren't tracked in one place, findings were hard to find, and the same participants risked being recruited again and again across verticals.
In-person research had no policy, procedure or guidance at all. The team had been too small to need it, and COVID had paused in-person work.
What I built
Three programs, plus the operations underneath them.
UserTesting program
UserTesting already ran most moderated and unmoderated research, so I built tracking around it rather than replacing it. I co-developed the tracker with researchers, managers and designers in three parts: research setup, conducting research and deliverables, with rules, enablement FAQ sessions and deliberately light research oversight. It ran about 50 studies a quarter.
Templates that hand ownership back
Templates and guardrails, some built into UserTesting, plus an interview tracker linking each session to its participant details, interview guide and viewing link. Researchers and stakeholders could run their own work, and operations stepped in only for seat allocation about once a quarter.
Product Research Library
Built a research archive in Airtable covering UserTesting projects, larger studies and surveys. Anyone in research or design could submit through a simple form that filed work into the right category. Duplicate research and repeat questions dropped by at least 20% right away.
Dedicated panels by vertical
Set up separate participant panels for each product vertical, reducing overlap, over-compensation and panel fatigue.
In-person research program
Built everything in-person research needed: ethics, policy, documentation, data tracking, participant experience, recruitment and security, with building security and office operations handling rooms and access. A month-long focus group and panel ran under it before I left.
Inclusive research and accessibility
Launched an inclusive research and accessibility program so studies reached a broader range of participants.
Operations underneath
Owned the tooling ecosystem, budget and renewal negotiations, recruitment infrastructure, incentives and scheduling. Built onboarding programs and success metrics for product design and research, and laid the operational groundwork for research behind Intuit and Credit Karma's AI-assist products.
| Research setup | Conducting research | Deliverables | ||||
|---|---|---|---|---|---|---|
| Study | Team | Owner | Method | Status | Readout | In library |
| Onboarding flow test | Vertical A | Designer | Unmoderated | Complete | Linked | Yes |
| Pricing page interviews | Vertical B | Researcher | Moderated | In field | Pending | No |
| Navigation card sort | Vertical C | Designer | Card sort | Setup | Pending | No |
| Notification copy test | Vertical A | PM | Unmoderated | Complete | Linked | Yes |
Structure of the original tracker with placeholder data.
- Ethics and policy
- Rules for running research in person
- Documentation
- Procedures and guidance for researchers
- Recruitment
- Bringing participants on site
- Participant experience
- How participants are welcomed and hosted on site
- Data tracking
- How in-person session data is recorded and kept
- Security and access
- Rooms and building access, with security and office operations
Impact
- About 50 studies a quarter through the self-serve UserTesting program, with clear visibility into which teams used research most and which themes were rising.
- At least a 20% immediate drop in duplicate research and repeat questions once the library launched, with new requests grounded in past results instead of guesses.
- Recruitment about 40% faster and study turnaround about 30–35% faster.
- A program that scaled quickly without breaking, needing operations only about once a quarter.
- Credit Karma's first in-person research program, used for a month-long focus group and panel.