Senior Data Engineering Manager

About YipitData: YipitData is the market-leading data and analytics firm. We analyze billions of data points every day to provide accurate, detailed insights across industries, including consumer brands, technology, software, and healthcare. Our insights team uses proprietary technology to identify, license, clean, and analyze the data that many of the world’s largest investment funds and corporations depend on. We raised $475M from The Carlyle Group at a valuation over $1B, further accelerating our growth and market impact. We have been recognized multiple times as one of Inc’s Best Workplaces. As a fast-growing company backed by The Carlyle Group and Norwest Venture Partners, YipitData is driven by a people-first culture rooted in mastery, ownership, and transparency. With offices in New York, Austin, Miami, Denver, Mountain View, Seattle, Hong Kong, Shanghai, Beijing, Guangzhou, and Singapore, we continue to expand our reach and impact across global markets. This is a hybrid opportunity based in India. (Currently, the role is remote, but once we establish an office space in India, the role will become hybrid. The hybrid work schedule is flexible, meaning the team in collaboration with the manager will determine when they come into the office.) Standard IST working hours are permitted with the exception of 2-3 days per week, when you will join meetings with the US and LatAm team. On these days, work hours will be between 2:30 - 10:30pm IST. (Please note that we allow for flexibility on the following days to make up for the previous day’s late work schedule!) Why You Should Apply NOW: We’re scaling fast and need a hands-on Data Engineering Manager to join our dynamic Data Engineering team who can both lead people and shape data architecture. The ideal candidate possesses 3+ years of managing data engineers and 5+ years of experience working with PySpark, Python is a must. Data Bricks/ Snow Apache Iceberg/ Apache Flink/ and various orchestration tools, ETL pipelines, and data modeling. As our Data Engineering Manager, you will own the data-orchestration strategy end-to-end. You’ll lead and mentor a team of engineers while researching, planning, and institutionalizing best practices that boost our pipeline performance, reliability, and cost-efficiency. This is a hands-on leadership role for someone who thrives on deep technical challenges, enjoys rolling up their sleeves to debug or design, and can chart a clear, forward-looking roadmap for various data engineering projects. As Our Data Engineer Manager, You Will: - Lead, mentor, and grow a team of data engineers working on large-scale distributed data systems. - Architect and oversee the development of end-to-end data solutions using AWS Data Services and Databricks. - Hire, onboard, and develop a high-performing team—1-on-1s, growth plans, and performance reviews. - Collaborate with cross-functional teams including data science, analytics, product, and business stakeholders to understand requirements and deliver impactful data products. - Drive best practices in data engineering, coding standards, version control, CI/CD, and monitoring. - Ensure high data quality, governance, and compliance with internal and external policies. - Optimize performance and cost efficiency of data infrastructure in the cloud. - Architect and evolve our data platform (batch & streaming) for scale, cost, and reliability. - Own the end-to-end vision and strategic roadmap for various projects. - Create documentation, architecture diagrams, and other training materials. - Translate product and analytics needs into a clear data engineering roadmap and OKRs. - Stay current with industry trends, emerging technologies, and apply them to improve system architecture and team capabilities. You Are Likely To Succeed If: You hold a Bachelor’s or Master’s degree in Computer Science, STEM, or a related technical discipline. 8+ years in data engineering (or adjacent), including 2-3+ years formally managing 1-3 engineers. Proven hands-on experience with: - Big Data ecosystems (Spark, Hive, Hadoop) - Databricks (including Delta Lake,, MLFlow, Unity Catalog) - Robust programming experience in Python and PySpark. - Deep understanding of data modeling, ETL/ELT processes using Streaming, and performance tuning. - Experience managing Agile teams and delivering complex projects on time. - Excellent problem-solving, leadership, and communication skills. - Experience designing and implementing Agentic Models with Data Pipelines (Data Cleaning and creative Feature Engineering). - Practical LLM/RAG experience for search quality such as query understanding, semantic retrieval, reranker design. You are a self-starter who enjoys working with both internal and external stakeholders. Nice to have: Familiarity with ML/AI workflows and collaboration with data science teams. Nice to have: Experience with Airflow, Docker, or equivalent. We are committed to equal employment opportunity regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, marital status, disability, gender, gender identity or expression, or veteran status. We are proud to be an equal-opportunity employer. Job Applicant Privacy Notice Originally posted on Himalayas