About the Role
Anthropic’s Research team is looking for a Data Operations Manager to help us level up our human data operations and accelerate our ability to develop models. This person will work across all operational aspects of human data management, including scoping, coordinating, procuring and managing projects and deliverables, ensuring that human data collection operations are running smoothly and that the data collected has high utility, quality, diversity, and volume. They will build and maintain strong relationships with our Human Feedback (HF) resources and optimize how we do this work on an ongoing basis.
Responsibilities:
- Scope, coordinate, and procure data collection and labeling projects and identify and capitalize on opportunities to make them more efficient.
- Monitor ongoing projects to make sure that resources are directed appropriately, adjusting scope and instructions with new information.
- Develop and maintain strong relationships with external vendors and contractors, ensuring that our work together is high-value and low-friction.
- Execute on human data projects, doing activities such as writing labeling instructions, reviewing data, and making sure we have the right mix of quality, diversity, and volume of data, as well as figuring out how to improve the tooling or instructions to make things more efficient.
- Report out on projects, including dashboards and data analysis.
- Partner with Engineering to maintain and improve our human feedback interface.
- Develop and support the operations for an in-house team of human data workers.
You might be a good fit for this role if you:
- Are extremely detail-oriented and enjoy poring over information and data line-by-line.
- Have exceptional project and relationship management skills, with both internal and external parties.
- Have a strong understanding of how LLMs work and of prompt engineering.
- Have experience with data collection, labeling, and analysis.
- Have experience using tools like SQL, Python, Tableau, etc.
It is particularly valuable if you:
- Have experience with human data collection and labeling specific to large language models.
- Have managed a team of external contractors and/or vendors before.
- Have conceived of and launched new technical programs.
- Have worked with researchers and engineers.
