Medical AI Evaluator (Remote)

Role Title: Resident Medical Specialist (MD/DO) Role Type: Part-time Location: Remote What does day-to-day look like: Work with research teams to evaluate and improve how AI systems handle clinical reasoning. You'll design evaluation methods that test AI performance on real medical problems where clinical expertise makes the difference. 1. Design systematic evaluation frameworks for medical AI systems 2. Create clinical scenarios that test AI reasoning and decision-making capabilities 3. Build assessment methods that capture the nuance of clinical practice 4. Identify gaps in AI medical knowledge and reasoning 5. Collaborate with AI researchers to improve model performance Candidate Requirements: 1. Licensed physician in active clinical practice (any specialty) 2. Experience with clinical decision-making and evidence-based medicine 3. Interest in how AI can support clinical practice 4. Strong analytical and communication skills Why it Matters: Help shape the next generation of medical AI by ensuring these systems can handle the complexity of real clinical practice. Remote work with flexible scheduling around your clinical commitments. Engagement details: 1. Commitment: flexible engagement, remote working, up to 30 hrs/week 2. Duration: 1 month, with potential extensions based on performance and fit What You'll Do 1. Design systematic evaluation frameworks for medical AI systems 2. Create clinical scenarios that test AI reasoning and decision-making capabilities 3. Build assessment methods that capture the nuance of clinical practice 4. Identify gaps in AI medical knowledge and reasoning 5. Collaborate with AI researchers to improve model performance Apply Now!