Senior Data Engineer
InFynd
Coimbatore South, Coimbatore, IndiaPosted 2 days ago
Skill Required
IT JobsData EngineerDatabricksSparkPythondesignAzureCloudAPIsETLSQLandFulltime
Role overview
Design, develop, and maintain scalable ETL/ELT data pipelines using Azure Data Factory and Azure Databricks. Develop data ingestion and transformation workflows using Python, SQL, and PySpark. Work with Apache Spark and PySpark for large-scale distributed data processing. Implement batch processing, incremental loading, full loading, and Change Data Capture (CDC) mechanisms. Integrate data from multiple sources, including relational databases, APIs, and cloud storage.