Senior Software Engineer, Code Quality for AI Training (Python)
G2i
WorldwideremotePosted 1 month ago
Skill Required
Senior-AI-Software-EngineerSenior-Software-AI-EngineerSenior-Python-Developer-AI-MLSenior-AI-ML-Engineer-Code-Training-DataSenior-Python-Software-EngineerSenior-AI-ML-EngineerSenior-AI-Software-DeveloperSenior-AI-EngineerSenior-AI-ML-Data-EngineerSenior-Python-EngineerAI EngineerSoftware EngineerSoftware DeveloperPython DeveloperPythonAIEngineeringJavaScriptTypeScriptdesignOpenAPIandContract
Key highlights
- $100–$200/hour
- 10–20 hrs/week
- Start ASAP (through early May)
- Staff / Principal-level engineering experience required
- Contract role (not traditional engineering)
- Evaluating AI coding agents like OpenAI Codex and Claude Code
Role overview
Senior AI Interaction Evaluator (Codex / Claude Code) – a contract role focused on evaluating AI coding agent interactions rather than writing production code. The position assesses whether models like OpenAI Codex and Claude Code demonstrate engineering judgment, useful reasoning, and developer-appropriate interaction patterns. This is a 10–20 hour/week contract through early May with $100–$200/hour compensation.
Responsibilities
- Evaluate AI-generated coding interactions end-to-end
- Judge whether outputs are useful, correct at a high level, and aligned with strong engineering thinking
- Assess quality of explanations and reasoning, not just code
- Distinguish between different levels of response quality (e.g. what makes something a 2 vs 4)
- Provide clear, opinionated feedback on what worked, what didn't, and what felt "off" or misleading
- Help define what great looks like when interacting with tools like Cursor
Requirements
- Staff / Principal-level engineer (or equivalent experience)
- Strong background in TypeScript / JavaScript or Python
- Hands-on experience using OpenAI Codex and Claude Code
- Deep familiarity with modern AI-assisted dev workflows
- Able to evaluate code without needing to fully execute or deeply review every line
- Comfortable giving direct, opinionated feedback
- High bar for what "good engineering" looks like
Nice to have
- Experience with tools like Cursor or similar AI-first IDEs
- Prior exposure to prompt design or evaluation workflows
- Experience mentoring senior engineers or defining engineering standards
Additional details
- Contract position
- $100–$200/hour rate
- 10–20 hours/week
- Start ASAP (through early May with possible extension)
- Take-home evaluation exercise required
- One behavioral interview required
- Source originally posted on Himalayas
- Focus is on engineering taste, not syntax correctness
- Role involves subjective but rigorous judgment about AI agent behavior