Posted today · be early
Senior Software Engineer, AI Training
G2i Inc.
WorldwideremotePosted today
Skill Required
Senior-AI-Software-EngineerSenior-Software-AI-EngineerSenior-AI-EngineerSenior-AI-Software-DeveloperSenior-AI-ML-EngineerSenior-AI-ML-Engineer-Code-Training-DataSoftware-EngineerAI EngineerSoftware EngineerSoftware DeveloperAIEngineeringJavaScriptTypeScriptwrittenPythondesignOpenAPIandContract
Key highlights
- Pay rate: $100-$200/hour
- Experience Level: Senior (SR+), Staff, or Principal
- Key Benefit: Flexible 10-20 hours/week schedule
- Notable Requirement: Live with OpenAI Codex, Claude Code, or Cursor
- Work Style: Remote worldwide (subject to regional support)
- Process: No interview, only a take-home with Loom walkthrough
Role overview
We're looking for a highly senior (SR+) software engineer to help evaluate the quality of interactions with modern coding agents like OpenAI Codex and Claude Code. This is not a traditional engineering role involving production code; instead, you will assess whether models think like great engineers by judging the usefulness, reasoning, and engineering judgment in AI-generated responses. The focus is on engineering taste: determining if outputs are helpful, build trust, and reflect strong engineering standards rather than just syntax correctness.
Responsibilities
- Assess how AI coding agents behave in real-world scenarios
- Judge whether AI-generated coding interactions are useful, correct at a high level, and aligned with how a strong engineer would think
- Evaluate whether responses make sense, if preambles and reasoning are useful, and if the interaction feels right to an experienced developer
- Assess the quality of explanations and reasoning, not just the code
- Distinguish between levels of response quality (e.g., what makes something a 2 vs a 4)
- Provide clear, opinionated written feedback on what worked, what didn't, and what felt off or misleading
- Help define what 'great' looks like when working with tools like Cursor, Codex, and Claude Code
- Determine if explanations are helpful or just technically correct, and if the model is guiding the user well or just dumping output
- Record short video explanations as part of the feedback process
Requirements
- Senior, Staff, or Principal-level engineer (or equivalent experience)
- Strong background in TypeScript/JavaScript or Python
- Hands-on experience with at least one of OpenAI Codex, Claude Code, or Cursor
- Deep familiarity with modern AI-assisted dev workflows
- Ability to evaluate code without executing it or reviewing every line
- Strong written and spoken English (B2 or above) to write detailed feedback and record short video explanations
- Comfortable giving direct, opinionated feedback
- Ability to make subjective but rigorous judgments and explain them clearly
- High bar for what good engineering looks like
Nice to have
- Prior exposure to prompt design or evaluation workflows
- Experience mentoring senior engineers or defining engineering standards
Benefits
- Flexible working hours
- Access to new project opportunities for evaluators who perform well
Additional details
- Contract type
- Remote work from anywhere in the world
- Pay rate of $100-$200/hour depending on experience and location
- Time commitment of 10-20 hours per week
- Eligibility may vary depending on location, as some regions may not be supported by the client
- Ongoing duration with individual projects running for approximately two weeks to a few months
- Start date is as soon as the candidate clears the take-home and a project has an open seat
- Hiring process consists of one take-home evaluation exercise with a recorded Loom walkthrough, with no interview
- Originally posted on Himalayas
- Role involves evaluating if interactions build or erode trust
- Role is not a traditional engineering role and does not involve writing production code
