Data Engineer (4 - 5 Years)

Job Title: Data Engineer Location: Coimbatore, Tamil Nadu, India Experience: 4–5 Years Employment Type: Full-Time   About the Role We are looking for an experienced Data Engineer to design, build, and maintain scalable data pipelines and infrastructure. The ideal candidate will have strong hands-on experience with cloud-based data platforms (AWS/Azure), big data processing frameworks, and API development, along with a solid foundation in SQL, Python, and data warehousing concepts.   Key Responsibilities Design, develop, and maintain robust ETL/ELT pipelines to ingest, transform, and load data from multiple sources Build and optimize data models for analytics, reporting, and downstream consumption Develop and manage big data processing workflows using PySpark and Databricks Design, document, and expose REST APIs using OpenAPI (Swagger) specifications Build and manage cloud-based data solutions across AWS and Azure Create and maintain dashboards and reports using Power BI and Amazon QuickSight Write efficient, well-documented SQL queries for data extraction, transformation, and validation Collaborate with data analysts, data scientists, and business stakeholders to understand data requirements Ensure data quality, integrity, and security across pipelines and storage systems Use Git/GitHub for version control and participate in code reviews Test and validate APIs using Postman and Swagger UI Document technical processes, data flows, and architecture Core Technical Skills Programming & Querying: SQL, Python Big Data Processing: PySpark, Databricks Cloud Platforms: AWS, Azure Data Architecture: Data Warehousing, ETL/ELT, Data Modeling APIs: REST APIs, OpenAPI (Swagger) Visualization & Reporting: Power BI Tools & Technologies AWS Services: Glue, S3, Athena, DynamoDB, Lambda, API Gateway, IAM, QuickSight API Development & Testing: Postman, Swagger UI Development & Version Control: VS Code, Git/GitHub, Jupyter Notebook Reporting & Analysis: Power BI Desktop, Microsoft Excel Qualifications Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field 4–5 years of hands-on experience in data engineering or a similar role Strong understanding of cloud data architecture (AWS and/or Azure) Experience with big data processing frameworks (PySpark, Databricks) Proven experience designing and consuming REST APIs Strong problem-solving skills and attention to detail Excellent communication and collaboration skills Good to Have AWS/Azure certifications Experience with CI/CD pipelines Exposure to Agile/Scrum methodologies Knowledge of data governance and security best practices Show more