Test Lead (AI Validation & Integration)

Digital Forms

WorldwideremotePosted 8 days ago

Skill Required

QA-EngineeringAutomation-TestingAI-TestingSalesforce-TestingQA-EngineerLead-AI-Test-Automation-ArchitectQA-Test-LeadTest-Automation-LeadAI-ML-Testing-EngineerAI-Platform-Test-EngineerAutomation-Test-LeadLead-Automation-TesterLead-Test-EngineerAISalesforceautomationdevelopingdesigningsecuritywrittenetc.)TestNGPythonPytestdesignCI/CDISO 27001JavaJiraAWSandGenerative AIManual TestingCICDJavaScriptMachine LearningContract

Key highlights

Lead QA and AI validation for a regulated AI + Salesforce project, owning strategy and hands-on testing across E2E, integration and NFR with CI/CD and close client collaboration.
New project for established client relationship, moving from Discovery into Delivery phase
Highly regulated industry with complex AI + Salesforce integration requirements
Full-time role with required overlap (14:00-18:00) for USA client collaboration

Role overview

We are looking for an experienced Test Lead to drive the Quality and Validation strategy for a high-impact solution in a highly regulated industry, involving AI components and Salesforce integration. This is a new project for a client with whom we have already established collaboration during other projects. We are finalizing the Discovery Phase for this project and moving into Delivery. The solution involves developing a data validation and automation engine that processes structured and unstructured inputs against a complex logic framework, utilizing AI models to interpret user-submitted data, identify discrepancies and missing information, and map results to structured output formats. Key components include automated ingestion, rule-based validation logic, and integration with existing enterprise architecture.

Responsibilities

Own the QA Strategy: Design, plan and develop the comprehensive Testing and AI Validation Strategy, covering Functional Unit, E2E, Integration, Regression, and User Acceptance Testing (UAT).
Address NFR and align against standards (e.g. GDPR, HIPAA, HITRUST, etc.)
AI Validation Logic: Define validation plans for AI components, specifically determining what constitutes "correct" output, setting success thresholds, and establishing evaluation methods (including handling hallucinations and grounding rules).
Environment & Data: Create detailed plans for Test Environments (Dev/Test/Prod) and define the Test Data Strategy to ensure sufficient coverage for standard scenarios, edge cases, and ingestion failures.
Hands-on Testing: Execute functional, regression, exploratory, performance, and load tests.
Integration Coordination: Coordinate "Joint End-to-End Validation" efforts, collaborating with the client team to manage testing of Salesforce integration
Pipeline Integration: Integrate testing activities into the CI/CD pipeline for continuous quality assurance
Process Management: Manage Jira test case structures aligned to user stories and acceptance criteria
Metrics: Provide regular reporting, tracking key QA metrics such as defect ratios

Requirements

Fluent English
QA Strategy: Experience defining end-to-end testing strategies for complex solutions
End-to-End Testing: Experience in designing and executing E2E tests for complex, integrated systems using any modern testing technology
AI/ML/LLM Testing Experience: Proven ability to validate AI models
Test Data Management: Experience strategizing and managing test data across multiple environments (Dev/Test/Prod)
Traceability mindset: linking requirements → test cases → evidence → defects → release decisions.
Experience testing NFRs: performance, reliability, auditability, security basics.
Strong understanding of privacy and sensitive data handling in testing (PII/PHI minimization, masking/anonymization)
Hands-on automation experience (at least one strong stack): e.g., Python/Java/JS + frameworks (pytest/JUnit/etc.)
QA Metrics: Ability to track and report on specific QA metrics, such as defect ratios
Good communication skills (written and verbal)
Jira Proficiency
Full time availability, with overlap during 14:00-18:00 - to ensure seamless collaboration with USA based client