Posted today · be early

Data Engineer

Tech Mahindra

Hyderabad, Telangana, IndiaPosted today

FulltimeData EngineerETLDatabricksObservabilitySparkAzureUnitySQL

Key highlights

Design, implement, and maintain data pipelines for data ingestion, processing, and transformation in Azure.
Work together with data scientists and analysts to understand the needs for data and create effective data workflows.
Create and maintain data storage solutions including Azure SQL Database, Azure Data Lake, and Azure Blob Storage.
Utilize Azure Databricks to create and maintain ETL (Extract, Transform, Load) operations.
Implement data validation and cleansing procedures to ensure the quality, integrity, and dependability of the data.
Improve the scalability, efficiency, and cost‑effectiveness of data pipelines.
Monitor and resolve data pipeline problems to guarantee consistency and availability of the data.

Azure Databricks with fact & dimension tables, Spark SQL, PySpark, Autoloader, Unity Catalog, and JSON file format – mandatory.
Hands‑on experience in Azure Databricks and Medallion Architecture.
7–9 years of experience with Azure Databricks technologies.
Experience in Medallion Architecture – mandatory.
Good understanding of ETL, BI, and data‑warehouse technologies.
Hands‑on experience with JSON or XML files as sources.
Experience with Autoloader for incremental loading and explode options for flattening data.
Experience loading Delta Live Tables.
Experience in PySpark or Spark using DataFrames.
Good oral and written communication skills.