Skip to main content
Back to jobs

Labeler / Annotator - AI Response Evaluation (Japanese)

External
bpcs logoBpcs · Worldwide
Full-timeRemote1mo ago30+ days old, may be filled
CompliancePayroll
Cover LetterConnect

Prepare for this interview

Elite

AI-generated questions, company research, and talking points tailored to this role

About the role

We are seeking a detail-oriented Labeler / Annotator to evaluate responses generated by AI systems in Japanese . This role focuses on side-by-side (SBS) evaluation of outputs from different AI models across real-world scenarios. You will play a key role in improving how AI systems understand and communicate in Japanese. This is not a translation role. It is an evaluation and analysis role requiring strong judgment and attention to detail.

Responsibilities

  • You will evaluate AI responses across scenarios such as:
  • General-purpose Q&A
  • Web search results
  • File-based and image-based responses
  • Image and file generation tasks
  • Single-turn and multi-turn conversations
  • Perform side-by-side (SBS) comparisons of AI-generated responses
  • Evaluate outputs based on:
  • Accuracy
  • Relevance
  • Clarity
  • Instruction-following
  • Identify nuances in tone, meaning, and cultural context across Japanese
  • Apply detailed, scenario-specific annotation guidelines
  • Maintain consistency and high-quality evaluations
  • Required Qualifications
  • Native or professional fluency in Japanese
  • Strong English reading comprehension (required for guidelines)
  • Prior experience with side-by-side (SBS) labeling or annotation
  • Excellent analytical thinking and attention to detail
  • Ability to quickly understand and apply structured evaluation frameworks

Requirements

  • Background in linguistics, translation, or localization
  • Experience in:
  • AI evaluation
  • Data annotation
  • Search relevance or content quality analysis

Benefits

We offer competitive compensation aligned with local markets and experience.Estimated hourly range: ARS 15,000 - ARS 17,000 Estimated monthly equivalent (full-time): ARS 2,400,000 - ARS 2,720,000Actual compensation will be determined based on experience, skills, and internal equity.Training & QualificationAll new hires will complete a structured onboarding program, including:Training sessions and practice exercisesCalibration against quality benchmarksQualification review before moving into production workBenefits & Employment StructureThis role is hired through an Employer of Record (EOR) partner, ensuring full compliance with local labor laws and payroll requirements.Employees receive locally compliant benefits, which may include:Statutory benefits (e.g., paid vacation and aguinaldo / Sueldo Anual Complementario (SAC))Social security contributions and healthcare coverage (Obra Social)Additional benefits aligned with local market practicesWhy Blueprint?Work on cutting-edge AI systemsBe part of a global, multilingual teamBuild highly transferable AI evaluation skillsContribute directly to improving real-world AI experiencesEqual Opportunity EmployerBlueprint Technologies is an equal opportunity employer. We consider all qualified applicants without regard to race, color, age, disability, gender identity, sexual orientation, religion, national origin, or any other protected status.Health insurancePaid time offEquity / stock options

Additional Information

Who is Blueprint Technologies ? We are a technology solutions firm headquartered in Bellevue, Washington, with a strong presence across the United States and an expanding footprint across Latin America (LATAM). Our teams are united by a shared passion for solving complex problems. At Blueprint, we use technology as a bridge between strategy and execution. Our people bring diverse perspectives, deep expertise, and real-world experience across industries to help organizations grow, transform, and innovate.

Your Match

How well this role fits your profile.

Company Intel

What employees say

Worked at bpcs? Share your experience

Interested in this role?

Apply on the company's website.

Cover LetterConnect