[Remote] Korean AI Model Rater

Note: The job is a remote job and is open to candidates in USA. Productive Playhouse is a global data company specializing in language and data services. They are seeking a Korean AI Model Rater to evaluate AI-generated responses and their integration with digital services, focusing on the accuracy and quality of these interactions.

Responsibilities

• Analyze and rate AI-generated responses and their associated tool outputs against a detailed set of quality guidelines
• Verify that the AI successfully integrates information from different digital services (APIs) as required by the prompt
• Evaluate the accuracy, completeness, and overall quality of the AI's response and tool execution
• Follow provided instructions to achieve task goals, with all evaluations recorded in English

Skills

• Advanced/Expert fluency in Korean language, as spoken in the South Korea region (ISO: [ko_kr])
• Strong comprehension of English (read and write)
• Strong conceptual knowledge of APIs and digital services (ability to understand how AI tools interact with external apps like search, maps, or calendar, without needing to write code)
• Ability to follow complex technical guidelines precisely
• All participants must have, or be willing to create, an Upwork account
• While participating in this project, adherence to the confidentiality terms outlined in the Upwork User Agreement is required
• Applicants must pass a quick skill verification assessment (MC, duration ~10 minutes)
• Work must be performed on participant's own devices
• You must use your own laptop, and smartphone (if required)
• A Gmail account may be required to access certain tools. If you do not have a Gmail account, you must be willing to create one
• Using AI during the assessment and work is STRICTLY forbidden, and results/inputs will be monitored to detect AI usage

Benefits

• Flexible schedule

Company Overview