Senior Data Engineer (AI/ML)
OpenTable
Skill Required
Key highlights
- Level Required: 5+ years
- Location: 100% remote across India
- Key benefit: Annual company-wide week off
- Notable requirement: Practical experience building applications using LLMs or Generative AI
Role overview
OpenTable, part of Booking Holdings, Inc., is looking for a Senior Data Engineer – AI/ML to help build the data and AI infrastructure powering their next generation of intelligent products and experiences. This role combines modern data engineering with Generative AI, designing scalable data platforms and pipelines while building production-grade solutions using LLMs, RAG, embeddings, vector search, and AI agents. The successful candidate will work closely with data scientists, ML engineers, software engineers, and product teams to turn AI capabilities into reliable, scalable production systems, helping build the data and AI foundation for intelligent products by combining large-scale data engineering, streaming systems, and modern Generative AI.
Responsibilities
- Design and build AI/LLM data pipelines supporting training, inference, evaluation, embeddings, and retrieval workloads.
- Build production-grade RAG systems, including ingestion, chunking, embedding generation, indexing, retrieval, reranking, and context construction.
- Develop AI applications using LLMs, structured outputs, function/tool calling, and agentic workflows.
- Build and optimize semantic search and vector retrieval systems.
- Develop frameworks for LLM evaluation, monitoring, tracing, quality measurement, latency, and cost optimization.
- Design scalable batch and streaming pipelines using Databricks, Apache Spark, Delta Lake, Snowflake, and Airflow.
- Build data products and platforms that make structured and unstructured enterprise data accessible to AI applications.
- Develop reliable ETL/ELT pipelines and optimize large-scale distributed workloads for performance and cost.
- Establish data quality, governance, lineage, security, and observability practices.
- Partner with ML and application engineering teams to move AI prototypes into production-ready systems.
Requirements
- 5+ years of experience in data engineering, software engineering, distributed systems, or a related field.
- Strong programming skills in Python and/or Scala/Java and advanced SQL.
- Hands-on experience with Databricks, Snowflake, Apache Spark, Delta Lake, and Airflow.
- Strong experience with cloud data platforms such as Snowflake and/or Databricks.
- Practical experience building applications using LLMs or Generative AI.
- Strong understanding of RAG architectures, embeddings, vector databases, semantic search, and retrieval systems.
- Familiarity with LLM concepts including prompting, structured outputs, tool calling, and model evaluation.
- Experience designing scalable, reliable, and observable production data systems.
Nice to have
- Deep experience designing large-scale data platforms, distributed processing systems, and complex data workflows.
- Strong experience with real-time and streaming data architectures, including Kafka, Spark Structured Streaming, or similar technologies.
- Experience building low-latency data pipelines and event-driven architectures.
- Experience designing complex multi-stage ETL/ELT and data orchestration workflows using Airflow or similar platforms.
- Experience optimizing Spark/Databricks workloads, including partitioning, clustering, caching, joins, and compute optimization.
- Experience building data platforms supporting both batch and real-time AI/ML workloads.
- Experience with LLM and AI evaluation frameworks, including automated evaluations, offline/online evaluation, quality metrics, and experimentation.
- Experience building evaluation datasets and pipelines for measuring LLM/RAG/agent quality, accuracy, relevance, latency, and cost.
- Experience with AI observability and tracing, including token usage, model performance, latency, failures, and production monitoring.
- Experience with LangGraph, LangChain, LlamaIndex, or similar AI orchestration frameworks.
- Experience with vector databases such as Qdrant, Pinecone, Weaviate, or Databricks Vector Search.
- Experience with Kafka, MLflow, Unity Catalog, Databricks Mosaic AI, or model-serving platforms.
- Experience building data quality, lineage, governance, and data/AI observability frameworks.
- Strong understanding of distributed systems, cloud architecture, APIs, CI/CD, and production operations.
Benefits
- Work from (almost) anywhere for up to 20 days per year
- Company-paid therapy sessions through SpringHealth
- Company-paid subscription to Headspace
- Annual company-wide week off a year
- Paid parental leave
- Generous paid vacation + time off for your birthday
- Paid volunteer time
- Development Dollars
- Leadership development
- Access to thousands of on-demand e-learnings
- Travel Discounts
- Employee Resource Groups
- Quarterly team offsites
- Tax optimisation options
- Generous health insurance
- Pension fund
Additional details
- This role is 100% remote across India location
- Schedule tailored to accommodate a global workforce operating across multiple time zones; may require communication via calls, Slack, or emails outside regular working hours to collaborate with international colleagues, respond to restaurant partners, and/or address urgent matters.
- OpenTable will always abide by and consider local laws and regulations.
- Commitment to creating an inclusive workplace; accommodations are available during the application, interview process, or on the job.
- Originally posted on Himalayas