Data Engineering offshore
Zensar
IndiaremotePosted 10 days ago
Skill Required
Data-EngineerBig-Data-EngineerCloud-Data-EngineerData-Platform-EngineerData Engineerdata engineeringEngineeringQuery OptimizationData WarehousingDesign PatternsObservabilitySQL ServerDatabricksautomationanalyticsbuildingPythondesignSparkAzureCloudHiveInformaticaETLSQLandAIFulltime
Key highlights
- 6+ years of experience delivering enterprise-scale data engineering, data warehousing, or big data solutions.
- Strong hands-on experience with Apache Spark, Python, SQL and Databricks.
- Experience with streaming or near real-time ingestion patterns.
- Experience implementing data quality, governance, metadata, and lineage capabilities.
Role overview
As a Senior Data Engineer, you will design, develop, and support scalable batch and near real-time data pipelines, data products, and lakehouse-ready data assets using Spark, Python, SQL, Hive, Databricks, and cloud-native technologies. You will drive modernization initiatives across hybrid data environments and cloud, enable trusted, governed, and accessible enterprise data through implementation of data quality, lineage, and governance standards, and partner closely with cross-functional teams to deliver strategic business outcomes.
Responsibilities
- Design, develop, and support scalable batch and near real-time data pipelines, data products, and lakehouse-ready data assets using Spark, Python, SQL, Hive, Databricks, and cloud-native technologies.
- Build and optimize capabilities supporting the ONE Data Lake and Mastercard's unified data platform strategy.
- Develop ingestion, transformation, and orchestration solutions for batch and streaming/near real-time data patterns using technologies such as Cribl, SSIS, Apache Ozone, Cloudera Data Services, Hive, Spark, Azure, Microsoft Fabric, Databricks, and related ecosystem tools.
- Enable trusted, governed, and accessible enterprise data through the implementation of data quality, lineage, and governance standards.
- Partner closely with Product, TPM, Data Governance, Architect, and Engineering teams to deliver strategic business outcomes.
- Support AI, analytics, reporting, and decisioning use cases by delivering scalable and reliable data assets.
- Drive modernization initiatives across hybrid data environments and cloud.
- Improve operational efficiency through automation, monitoring, observability, and self-service capabilities.
- Troubleshoot complex platform and data issues, perform root cause analysis, and implement sustainable solutions.
- Evaluate emerging technologies and identify opportunities to advance Team ONE's data capabilities.
Requirements
- 6+ years of experience delivering enterprise-scale data engineering, data warehousing, or big data solutions.
- Strong hands-on experience with Apache Spark, Python, SQL and Databricks.
- Experience building and maintaining large-scale data pipelines and distributed data processing systems.
- Experience with streaming or near real-time ingestion patterns, including event/log data collection, routing, transformation, and delivery into enterprise data lake or lakehouse platforms.
- Experience with Hive, Apache Ozone, or similar data integration, Azure, Microsoft Fabric, and orchestration technologies.
- Strong understanding of data architecture, ETL/ELT design patterns, and modern data platform principles.
- Hands-on experience with Microsoft SQL Server, SSIS packages, stored procedures, SQL performance tuning, and relational database concepts.
- Experience implementing data quality, governance, metadata, and lineage capabilities.
- Proven ability to analyze complex production issues, perform root cause analysis, and implement long-term solutions.
- Strong communication and collaboration skills with the ability to work effectively across technical and business teams.
- Demonstrated ability to manage competing priorities in a fast-paced environment.
- Strong ownership mindset and commitment to delivering high-quality solutions.
Additional details
- Originally posted on Himalayas