We are seeking a PySpark Data Engineer with strong experience in building scalable data pipelines, big data processing, and cloud-based data platforms. The ideal candidate will have expertise in PySpark, Databricks, SQL, and modern data engineering practices to support enterprise analytics and data-driven initiatives.
Responsibilities
- Design, develop, and maintain ETL/ELT pipelines using PySpark and Databricks.
- Process and transform large-scale structured and unstructured datasets.
- Build scalable batch and real-time data processing solutions.
- Optimize Spark jobs for performance, scalability, and cost efficiency.
- Implement data quality, governance, and monitoring frameworks.
- Integrate data from multiple enterprise and cloud sources.
- Collaborate with business, analytics, and architecture teams to deliver data solutions.
- Support CI/CD, deployment automation, and production troubleshooting.
Requirements
- Strong hands-on experience in PySpark, Python, and SQL
- Expertise with Azure Databricks or similar Spark platforms
- Experience in designing and optimizing data pipelines
- Knowledge of Spark SQL, DataFrames, UDFs, performance tuning
- Experience with Azure Data Factory (ADF) and Azure Data Lake
- Strong understanding of ETL/ELT concepts and data modeling
Additional details
- Hiring Company: TCS (Tata Consultancy Services)
- Role: PySpark Data Engineer
- Interview Location: Face to Face Interview on 8th August at Tata Consultancy Services Limited, Gitanjali Park Campus, IT/ITES SEZ, Plot- IIF / 3, (Block K - Ground Floor) Action Area - II, New Town, Rajarhat, Kolkata - 700156, West Bengal
Zobhira
