Lead Engineer - Data
Ollion
IndiaremotePosted 1 month ago
Skill Required
Data-EngineeringLead-Data-EngineerTechnical-LeadData-ArchitectureCloud-EngineerLead-EngineerLead-Data-EngineeringData-Engineering-LeadData EngineerQuery OptimizationETLKafkaGCPEngineeringDatabricksR ProgramminganalyticalSnowflakedesigningsecuritybuildingTestNGAzurePythondesignScalaAgileCloudISO 27001JavaSQLAWSGitFulltime
Key highlights
- Minimum 10 years of experience in Data Engineering or related roles.
- Minimum 2+ years in a Technical Lead (Data) capacity, with no management responsibilities.
- Proficiency in Python and SQL and experience designing cloud data pipelines with AWS, GCP, or Azure.
- Hands‑on experience with enterprise‑grade data lake solutions, Delta Lake, and real‑time streaming (Kafka or Kinesis).
- Relevant cloud certifications such as AWS Certified Data Analytics or Google Cloud Professional Data Engineer required.
- Strong understanding of data governance, privacy regulations (GDPR, CCPA, HIPAA) and security best practices.
Role overview
Responsibilities
- Technically Lead and mentor a team of data engineers, fostering growth, collaboration, and technical excellence across project delivery.
- Architect, design, and implement scalable data pipelines and processing systems using modern cloud services (AWS, GCP, Azure).
- Establish and enforce data engineering best practices, standards, and governance frameworks.
- Collaborate with business stakeholders to understand data needs and deliver actionable, scalable solutions.
- Drive the technical vision and roadmap for data infrastructure and platforms.
- Optimize data workflows for performance, reliability, and cost efficiency.
- Provide technical leadership in data modeling and ETL/ELT design.
- Conduct code reviews, ensuring scalable and maintainable solutions.
- Support teams and stakeholders with data‑related problem‑solving.
- Communicate technical concepts effectively to technical and non‑technical audiences.
- Design and lead discovery sessions with clients and internal teams for requirement gathering and architecture alignment.
- Automate and orchestrate complex data pipelines, reducing manual effort.
- Develop and maintain high‑quality documentation of architectures, requirements, and processes.
- Support pre‑sales engagements, including technical solutioning and proposal development.
- Create reusable frameworks and artifacts to accelerate delivery.
Requirements
- Minimum 10 years of experience in Data Engineering or related roles.
- At least 2+ years in a Technical Lead (Data) capacity.
- 100% hands‑on capability required; not a management position.
- Proficiency in Python and SQL, with proven experience in developing scalable data solutions.
- Strong skills in writing clean, testable, production‑grade code for data transformation and orchestration.
- Experience with version control systems like Git and collaborative development practices.
- Proven experience in designing and implementing enterprise‑grade data lake solutions.
- Hands‑on experience with Cloud Data Engineering (AWS/GCP), Databricks, Delta Lake, and lakehouse architectures.
- Strong understanding of data warehouse concepts, including star and snowflake schemas.
- Expertise in query performance tuning, data partitioning, compression, and storage optimization.
- Understanding of data governance, access control, and security frameworks.
- Familiarity with medallion architecture or equivalent multi‑tier data processing frameworks.
- Experience with real‑time data streaming technologies such as Apache Kafka or AWS Kinesis.
- 2+ years of technical leadership experience.
- Demonstrated ability to mentor junior engineers and foster team capability growth.
- Excellent communication and collaboration skills across cross‑functional teams.
- Strong problem‑solving abilities, business acumen, and analytical thinking.
- Ability to manage multiple priorities in dynamic, fast‑paced environments.
- Customer‑focused mindset with strong stakeholder engagement skills.
- Experience in Agile methodologies and leading sprint planning and retrospectives.
- Expertise in data pipeline architectures, data integration patterns, and orchestration tools.
- Experience with data validation, testing frameworks, and data quality management.
- Knowledge of data privacy, compliance, and security best practices.
- Experience in connecting data pipelines to visualization layers for real‑time analytics.
- Understanding of semantic layer design and self‑service data modeling.
- Experience in end‑to‑end orchestration of pipelines from source systems to reporting tools.
- Proficiency in data modeling techniques and database design principles.
- Relevant cloud certifications (e.g., AWS Certified Data Analytics or Google Cloud Professional Data Engineer).
- Understanding of industry‑specific data regulations (e.g., GDPR, CCPA, HIPAA) as applicable.
- Knowledge of emerging trends in data engineering, such as data mesh and data fabric architectures.
Nice to have
- Familiarity with Scala or Java, particularly for big data processing frameworks.
Additional details
- Ollion is an equal opportunity employer. We celebrate diversity and we are committed to creating an inclusive environment for all employees. Ollion does not discriminate in employment on the basis of race, color, religion, sex (including pregnancy and gender identity), national origin, political affiliation, sexual orientation, marital status, disability, genetic information, age, membership in an employee organization, parental status, military service or other non‑merit factor.
- Let’s be honest: there are lots of people out there doing what we do. We’re just not convinced they’re doing it right. Businesses are hungry for innovation and opportunity, but not at the cost of their independence. At Ollion, we’ve connected companies and capabilities around the world to help ambitious organizations make the most of their transformation and leave the status quo in the dust.
- Innovation is risky. It demands bold steps and big questions, but that’s the price of making change. We’ve got our head in the cloud and two feet on the ground, channeling tech’s endless potential towards a single goal: making a world of difference. And we’re building a global team to do just that— a team capable of making game‑changing breakthroughs without ever losing sight of the people it will impact. This is more than consulting. This is the change you can be.
- At Ollion, we’re all in on your independence. Our teams are seasoned. Our solutions are straightforward—sometimes even groundbreaking. And our engagements? Exactly as long as you want them to be. We deliver fresh thinking and hard‑earned insight in a way that works for you and your customers, arming your organization with everything you need to make your transformation truly mean something.
- Progress matters more than process. Our global team of cloud‑native pros is all about creating new and better ways to work—not just by solving your tech challenges, but by using technology to solve your business challenges. We keep the formulas, frameworks, and ten‑point plans to a minimum, tackling your most pressing problems with a proprietary mix of good‑old‑fashioned ingenuity and refreshing humanity.
- Originally posted on Himalayas.