Research Scientist

AI Research Scientist Frontier AI $5 Million Raised SF Onsite






Ghost AI is working with a $5M-backed AI company building the benchmarks that determine how the world's leading AI models perform in the real world.





Their work spans finance, law, healthcare, software and more, with evaluations developed in-house and used to rigorously test frontier models.





Now they're looking for a Research Scientist to help push this work further.





Why this opportunity?





This isn't a role where you'll simply run existing benchmarks.





You'll have the opportunity to build new evaluation methodologies from scratch, work directly with cutting-edge AI models, and help shape how the next generation of AI systems are tested before they're deployed in the real world.





The company was founded out of Stanford NLP research, with a team whose backgrounds include
NVIDIA, Meta, Microsoft, Palantir and Hudson River Trading, and
300+ citations across their published research.





They're growing quickly — the team has doubled in the last six months.





What you'll do






- Evaluate frontier models as they're released, including models such as Gemini and DeepSeek

- Design and build new AI benchmarks from the ground up
- Construct datasets, work with labelers and develop rigorous evaluation rubrics

- Develop and improve automated methods for evaluating generated text
- Work closely with engineers to implement and scale evaluation systems
- Collaborate with leading AI labs and enterprise customers to understand emerging evaluation needs

- Write technical reports and white papers around your research






One example of their work: Public Benefits Bench, developed with Code for America and the Center for Civic Futures, tested 459 scenarios across all 50 states and found that the best-performing model was correct only
68.1% of the time.





That's the kind of research that can have real-world consequences.





What they're looking for





Must have:






- 0–3 years of experience in applied AI/ML
- Research experience in AI/ML, LLMs, benchmarking or evaluation

- Strong Python skills and ability to write clean, maintainable code
- Experience with PyTorch, TensorFlow or similar deep learning frameworks
- Comfortable working in an engineering environment — Git, code reviews, development sprints, etc.

- Strong communication and collaboration skills






Strongly preferred:






- Experience building benchmarks or evaluation methodologies
- Experience at an AI startup, research lab or as an early-stage/founding engineer

- NLP or language-model research experience
- Master's or PhD in CS, ML or a related field
- Familiarity with LLM infrastructure and Python web frameworks such as Django or Flask






Most importantly: they're looking for someone who wants to
build and apply research, not someone whose primary motivation is publishing papers.





The team





The founding team brings experience from Stanford, Palantir, HRT, NVIDIA, Meta, Microsoft and Snapchat, alongside hundreds of research citations.





You'll work alongside a highly technical team with the resources to run
large-scale AI evaluation studies and collaborate with some of the most important AI labs in the industry.





What's on offer?






- Highly competitive salary + meaningful equity
- San Francisco — relocation/transportation support
- Health & dental insurance
- Lunch & dinner provided + snacks, coffee & drinks
- Unlimited PTO
- 401(k)
- Friday happy hours & regular team outings














If you're an early-career AI researcher who wants to work on problems that actually influence how frontier models are evaluated, this is worth a conversation.
👻