AI Research Scientist, Generative AI Evaluation Foundations, AI Judges team

The Generative AI Evaluation organization's mission is to measure GenAI models' progress (in particular Llama models) and accelerate the world's AI development. This implies building measurements and methodologies for these assessments. In particular, the AI judges team focuses on building evaluations relying on AI-assistance (e.g., LLM as judges, classifiers), covering a broad spectrum of metrics from helpfulness to safety. We are looking for a strong AI research scientist, with passion for Generative AI and in particular measurement/assessment of these cutting edge models. You'd join one of our projects and build methods to solve open challenges and unblock the execution and application of AI-assisted measurements to GenAI models, particularly Llama.We are looking for a team player, who thrives in ambiguity, quickly adapting to new problems and AI metrics from helpfulness to safety; preferred candidates have experience in doing applied research/evaluations on large scale GenAI models., 1. Conduct research on the metrics and methodologies to assess/evaluate GenAI models (in particular Llama models)
2. Collaborate with cross-functional teams to ensure that our GenAI models are assessed leveraging state of the art solutions, and potentially co-develop them with partner teams
3. Stay up-to-date with the latest developments in the field of Generative AI and their evaluations/AI-assisted evaluation in particular
4. Communicate research findings and recommendations to stakeholders and scientific community

AI Research Scientist, Generative AI Evaluation Foundations, AI Judges team Responsibilities, 5. PhD in the field of Deep learning, Machine Learning, Computer Vision, NLP, Computer Science or a related field.
6. Bachelor's degree in Computer Science, Computer Engineering, or a related field
7. Experience with conducting research and publishing in top tier peer-reviewed journals and AI conferences
8. Excellent communication and collaboration skills
9. Ability to work independently and as part of a team

Preferred Qualifications:

Preferred Qualifications:
10. Background and/or previous work in Generative AI
11. Background and/or previous work in evaluations of GenAI models
12. 3+ years of experience after PhD
13. Track record of setting technical direction, driving consensus and successful cross-functional partnership