Data Engineering Lead - Data Quality Systems
Firmable
IndiaremotePosted 13 days ago
Skill Required
Data-Engineering-LeadData-EngineerApplied-AI-EngineerData-Quality-EngineerMachine-Learning-EngineerLead-Data-EngineeringData-Quality-LeadLead-Data-EngineerSenior-Data-Quality-EngineerData-Engineering-Team-LeadSenior-Lead-Data-EngineeringData-Quality-EngineeringData Engineerdata engineeringEngineeringDatabricksSnowflakeRedshiftbuildingAirflowTestNGPythondesignCloudRustGenerative AIETLSQLAWSandAIFulltime
Key highlights
- Data Engineering Lead — Data Quality Systems role focused on building quality infrastructure for billions of records
- Minimum 7-10 years experience building production data systems in business-critical environments
- Competitive base plus meaningful equity compensation
- Ownership of the trust layer — every record sold passes through systems you architect
- Small team, real leverage — lead the engineers building the quality standard for 13 markets
Role overview
Firmable is the market-leading B2B sales intelligence platform in Asia Pacific, backed by leading investors and serving 2,000+ customers worldwide. They are scaling their successful model globally and seek a Data Engineering Lead — Data Quality Systems to build and maintain the quality infrastructure that ensures the integrity of company and people records across 13 markets. The role focuses on creating robust data quality systems that protect the trust layer for every record sold.
Responsibilities
- Lead a small team of Applied AI Engineers building systems that measure, verify, and enforce quality across billions of company and people records in 13 markets
- Architect the quality layer including verification pipelines, evaluation harnesses, LLM-based validators, anomaly detection, and release gates that control what data ships to customers
- Design and build verification, sampling, and scoring pipelines that run continuously over billions of rows across every market
- Create reusable frameworks such as harnesses, abstractions, and SKILL.md specifications that allow new quality checks to be developed quickly, executed efficiently, and implemented correctly
- Develop eval harnesses for LLM-based validation and extraction including labeled eval sets, precision/recall tracking, judge calibration, prompt versioning, and drift detection on vendor model updates
- Implement pre- and post-production release gates that block bad data before it reaches customers along with failure analysis and triage tooling
- Handle incident remediation by root-causing quality issues at scale, shipping fixes, and turning failures into permanent automated checks
- Lead 3–5 Applied AI Engineers providing technical direction, code review, pairing, and mentorship to grow them into engineers who own outcomes end to end
Requirements
- 7-10 years building production data systems in business-critical environments with a proven track record of shipping unattended, scalable systems that stay up
- Experience working with billions of rows and understanding how quality checks behave at that scale
- Built data quality systems and frameworks — validation engines, anomaly detection, scoring, sampling strategies, and reconciliation against ground truth
- Harness engineering experience designing eval or test harnesses that other engineers depend on, demonstrated in the repository
- Led engineers by directing a small team technically, reviewing code, and remaining hands-on during implementation
- Strong Python and advanced SQL skills with production-grade, performance-aware implementations and comfort working with concurrency and large-scale transformations
- Operate LLMs as production systems including management of eval sets, versioned prompts, logged traces, cost ceilings, and debugging judges for precision/recall
- Shipped real work using agentic IDEs such as Claude Code or Cursor as the default development mode
- Sharp judgement in deciding when to use deterministic rules versus LLM-based approaches depending on structural clarity and the need for semantic judgment
Nice to have
- Expertise in B2B data domains such as firmographics, people data, entity resolution, and cross-market registry matching
- Proficiency with cloud data platforms including Snowflake, Databricks, Redshift, and AWS for pipeline deployment
- Production-scale Airflow or equivalent orchestration experience
- Knowledge of vector databases, embeddings, or retrieval patterns for matching and deduplication tasks
- Startup or scaleup background where you defined data quality standards rather than inheriting them
Benefits
- Competitive base compensation
- Meaningful equity stake in the company
- Commitment as an equal opportunity employer supporting diverse talent
Additional details
- The organization operates as an AI-native company where agentic development, evals, traces, and AI-powered review are default modes of working, not experimental practices
- Teams follow a small senior structure with no layers, minimal process, and rapid releases moving toward daily cadence
- The role explicitly excludes traditional QA or testing responsibilities such as writing ETL test cases, application tests, or product acceptance tests
- Success is measured by maintaining trust in the data that powers Firmable's sales intelligence platform across 13 markets
- The position requires leading a team of 3–5 Applied AI Engineers with clear expectations around technical direction and hands-on contribution
- This role is not intended for candidates seeking primarily people-management responsibilities