Data Engineer (Databricks + AI)

Job Summary We are looking for an experienced AI / Data Engineer to design and build scalable data pipelines and AI-powered applications. The ideal candidate will combine strong data engineering fundamentals with hands-on experience in LLM frameworks, Retrieval-Augmented Generation (RAG), and API development to deliver production-grade AI and data solutions. Key Responsibilities Design, develop, and optimize scalable data pipelines using Azure/AWS/GCP Databricks and PySpark. Develop AI-powered applications using LLMs, LangChain, and LlamaIndex. Design and implement RAG (Retrieval-Augmented Generation) solutions using enterprise knowledge bases. Develop REST APIs using Python (FastAPI/Flask) for AI and data services. Collaborate with Data Scientists, ML Engineers, and Business stakeholders to deliver AI-driven solutions. Optimize model performance, prompt engineering, and inference pipelines. Follow DevOps and MLOps best practices for deployment, monitoring, and maintenance. Required Skills & Qualifications 3+ years of experience in data engineering, with strong expertise in AWS/Azure/GCP Databricks and PySpark/Python. Proven experience building and maintaining ETL/ELT pipelines at scale. Hands-on experience with LLM frameworks such as LangChain and LlamaIndex. Practical knowledge of RAG architecture and enterprise knowledge base integration. Strong Python skills, including API development using FastAPI or Flask. Familiarity with prompt engineering and inference pipeline optimization. Understanding of DevOps/MLOps practices (CI/CD, model deployment, monitoring). Strong communication and collaboration skills to work across cross-functional teams. Skills: rag architecture,langchain & llamaindex,api development,devops/mlops,databricks + pyspark/python,etl/elt pipeline development Show more