In this role candidate will be responsible for understanding complex data sets, managing multiple disparate data sources, and implementing strong validation processes to ensure data quality. The ideal candidate will have a strong background in python, ETL, pyspark, spark, sql, Amazon Web Services, and Databricks.
Responsibilities
- Develop and maintain ETL pipelines to support data processing, analytics, and visualization needs.
- Analyze large, complex data sets to identify patterns, trends, and insights.
- Collaborate with cross-functional teams to understand business requirements and provide data-driven solutions.
- Manage multiple disparate data sources and integrate them into a cohesive data platform and dashboards.
- Develop and implement validation processes to ensure data accuracy and consistency.
- Utilize pyspark, spark, sql, and Databricks to optimize data processing and querying performance.
Requirements
- 4 to 10 Years experience in PySpark and Data bricks
- Strong background in python, ETL, pyspark, spark, sql, Amazon Web Services, and Databricks
Additional details
- Position: Data Engineer
- Location: Hyderabad
Zobhira
