We are seeking a Data Engineer with 3–5 years of hands-on experience in Spark and SparkSQL, strong SQL expertise, and solid batch processing experience. The role involves building and optimizing data pipelines to support analytics and business needs, ensuring data reliability and performance, and collaborating effectively with cross-functional teams.
Responsibilities
- Building and optimizing data pipelines to support analytics and business needs
- Ensuring data reliability and performance
- Collaborating effectively with cross-functional teams
- Participating in code reviews
- Participating in design discussions
- Participating in agile delivery processes
- Performing ingestion, transformation, and loading (ETL/ELT) processes
- Managing scheduling, dependencies, failure recovery, and performance tuning for batch data processing
- Applying problem-solving and debugging skills to address pipeline performance and data quality
Requirements
- 3–5 years of hands-on experience with Apache Spark (especially Spark SQL and batch processing)
- Strong SQL expertise — ability to write efficient, optimized, and complex SQL queries for large datasets
- Solid understanding of data pipeline fundamentals, including ingestion, transformation, and loading (ETL/ELT) processes
- Experience with batch data processing (scheduling, dependencies, failure recovery, performance tuning)
- Knowledge of data warehousing concepts and familiarity with relational databases
- Experience working with structured and semi-structured data (CSV, Parquet, JSON, etc.)
- Strong problem-solving and debugging skills, particularly around pipeline performance and data quality
- Good project experience
Nice to have
- Python is a plus (not required)
Additional details
- Greetings from Flexton!
- Hope you are doing great today!
- One of my clients is looking for Data Engineer@ Bangolore, India please share me your updated resume and desire rate for this position.
Zobhira
