Speedrun Talent Network
Senior AI Evaluation Product Manager
- Location
- Hayes Valley, San Francisco
- Employment
- Full Time
Job summary
Hercules is seeking a senior or staff level product manager to own evals for our AI coding agent. You will own Hercules Bench offline evals and Hercules online evals, guiding evaluation design and data-driven improvements. You will leverage our 500k users, 1M code repos, and millions of chat threads to craft effective evals while anonymizing data to protect privacy. We ship, strive for excellence, and stay hungry for impact. J-18808-Ljbffr
SalaryCheck the source listing for salary
General interview preparation
General preparation guidance from MiPrep; these interview stages are not confirmed for this job.
- Recruiter phone screen - typically 20-40 min
- Technical screen - 1 coding problem or take-home; format varies by company
- Onsite / virtual loop - typically 4-6 rounds over 2-3 weeks; exact split of coding, systems design, and behavioral varies by role and location