Remote - anywhere is the US.
Annotation Judge to join Human In The Loop Operations team and contribute to growing set of Natural Language Processing (NLP) projects. The Annotation Judge will play a crucial role in fine-tuning and calibrating AI/ML processes that handle verbal interactions.
You will work directly with experienced Operations Managers and Natural Language experts. You will be responsible for ensuring the quality of a number of possible elements related to our projects, from the accuracy and humanlike traits of automated chatbots answers, to the classification of elements of a conversation annotated automatically or the review of search results based on understanding of complex users queries.
You will play an essential role in defining clear guidelines which may be used by non-native English speakers. Your attention to detail and natural curiosity for language subtleties will be essential for the long term scaling of our solutions.
Is a responsible and reliable self-starter who can be comfortable with ambiguity.
Major responsibilities:
• Review and/or evaluate annotated data to ensure it meets quality standards and project requirements.
• Provide feedback to ML/NLP engineers to improve accuracy and overall quality
• Collaborate with internal teams to optimize model performance based on annotated data.
• Help develop and maintain annotation guidelines and best practices.
• Provide training data to ML/NLP engineers to establish Ground Truth data.
• Provide input/examples on Training content.
• Identify gaps in training content and suggest improvements.
BASIC QUALIFICATIONS
• Bachelor's degree in English, Linguistics or a related field.
• Excellent attention to detail and analytical skills.
• Ability to work effectively in a collaborative team environment.
• Experience with tools and software used for NLP annotation is a plus.
New Grads welcome to apply - but must have work examples
PREFERRED QUALIFICATIONS
• Experience or demonstrated interest in linguistics
• Experience working on NLP projects with a focus on chatbots, text classification, and named entity recognition
• Experience with copywriting, UX authoring, content development around voice and brand guidelines.
• Experience with Machine-generated Speech evaluation, Prompt Engineering, Search Results Evaluation, Search Query Categorization
• Experience following complex processes
• Ability to work both independently and with a team to achieve a goal.Annot