Lead Data Engineer (Contract, Full-Time) [HR208]
Smart Working
IndiaremotePosted 2 months ago
Skill Required
Lead-Data-EngineerData-EngineeringData-Infrastructure-EngineeringAI-and-Data-EngineeringData-ArchitectureSenior-Lead-Data-EngineeringData-EngineerLead-Data-Engineer-JobsData-Engineering-LeadLead-Data-EngineeringLead-Data-and-AI-Platform-EngineerSenior-Data-EngineeringTech-Lead-Data-EngineeringData EngineerMachine Learningdata engineeringAirflowElasticsearchObservabilityEngineeringR ProgrammingPostgreSQLdesigninganalyticssecuritybuildingMongoDBPythonPandasdesignSparkKafkaMySQLAzureCloudjQueryAWSGCPandFulltime
Key highlights
- 7+ years of professional data engineering experience required.
- Remote‑first, fully remote opportunity with a supportive community.
- Founding data hire – shape the data team’s culture, standards, and hiring bar.
- Work with cutting‑edge technologies such as vector databases, real‑time pipelines, and AI platforms.
- Direct partnership with founders and product leadership.
- Role focuses on building data infrastructure for an AI assistant that automates property‑management operations.
Role overview
As a Lead Data Engineer, you will build and lead the data infrastructure powering an intelligent AI assistant that automates operational workloads for property managers, letting agents and build-to-rent teams. The platform manages communications, compliance, maintenance coordination and scheduling, helping property businesses operate with the efficiency of modern digital platforms.
Responsibilities
- Architect and build scalable data pipelines and infrastructure to support AI and product systems.
- Design and maintain data ingestion, transformation and storage architectures for operational and AI workloads.
- Develop and manage batch and real-time data pipelines.
- Build and optimise systems for vector search, retrieval and machine learning data pipelines.
- Ensure data reliability, security and governance across the platform.
- Collaborate with AI and backend engineering teams to support training, inference and product features.
- Implement monitoring, observability and data quality frameworks.
- Optimise the performance of large‑scale datasets and query systems.
- Contribute to technical architecture decisions and long‑term data strategy.
- Act as the founding data hire, defining culture, standards and the hiring bar for the data function as it scales.
- Partner directly with founders and product leadership to translate data capabilities into product decisions.
Requirements
- 7+ years of professional experience, with the majority of that experience in dedicated data engineering roles.
- Strong experience designing and building data pipelines and distributed data systems.
- Experience working with relational databases, with PostgreSQL preferred, although MySQL or similar is acceptable.
- Experience working with NoSQL databases.
- Experience with vector databases used in modern AI systems.
- Strong programming experience in Python.
- Demonstrated ability to make and justify architectural decisions, rather than only implementing them.
- Experience building scalable backend systems.
- Experience designing data models and storage architectures.
- Strong understanding of data processing performance and optimisation.
Nice to have
- Experience with some of the following data frameworks and infrastructure technologies is highly desirable: Apache Spark, Apache Airflow, Kafka, and Elasticsearch or OpenSearch.
- Experience with relevant database technologies is highly desirable, including PostgreSQL, MongoDB, and vector databases such as Qdrant, Milvus or pgvector.
- Experience with Python data‑processing libraries such as Pandas or Polars is highly desirable.
- Experience working on AI or machine learning platforms.
- Familiarity with stream processing and event‑driven architectures.
- Experience with cloud infrastructure such as GCP, AWS or Azure.
- Experience working in high‑growth startups or early‑stage companies.
Additional details
- About Smart Working: At Smart Working, we believe your job should not only look right on paper but also feel right every day. This isn’t just another remote opportunity - it’s about finding where you truly belong, no matter where you are.
- From day one, you’re welcomed into a genuine community that values your growth and well‑being.
- Our mission is to break down geographic barriers and connect skilled professionals with outstanding global teams and products for full‑time, long‑term roles.
- We help you discover meaningful work with teams that invest in your success, where you’re empowered to grow personally and professionally.
- Join one of the highest‑rated workplaces on Glassdoor and experience what it means to thrive in a truly remote‑first world.
- Originally posted on Himalayas.