Senior Spark QA Engineer

InfraCloud

IndiaremotePosted 18 days ago

Skill Required

QA-EngineeringSpark-QA-EngineerPerformance-TestingData-Engineering-QAETL-TestingSenior-QA-EngineeringSenior-QA-TesterSenior-QA-AnalystSenior-Quality-EngineerSenior-Product-QASenior-QA-Automation-EngineerSoftware-Quality-EngineerQA EngineerSparkSystem DesignJMeterCD pipelinesDatabricksKubernetesautomationTestNGPythonCI/CDCloudJavaETLSQLandCICDFulltime

Key highlights

5+ years of QA/testing experience
Strong expertise in functional and performance testing of Apache Spark
Strong understanding of Spark architecture and optimization techniques
Experience with Kubernetes and cloud‑based Spark environments
Proficiency in Python or Java
Experience with automation frameworks and CI/CD pipelines

Role overview

We are looking for a Senior Spark QA Engineer with strong expertise in functional and performance testing of Apache Spark applications and distributed data platforms.

Responsibilities

Perform manual and automated testing of Spark jobs, Spark SQL queries, and ETL pipelines.
Execute functional, scalability, and performance testing for Spark workloads.
Set up and manage Spark clusters on Standalone, YARN, Kubernetes, Mesos, Databricks, EMR, and Dataproc.
Conduct benchmarking, validation, and performance analysis of Spark applications.
Identify bottlenecks and troubleshoot distributed system issues.
Lead QA initiatives and mentor team members.

Requirements

5+ years of QA/testing experience with strong hands‑on expertise in Apache Spark.
Experience in functional and performance testing of distributed systems.
Strong understanding of Spark architecture and optimization techniques.
Experience with Kubernetes and cloud‑based Spark environments.
Proficiency in Python or Java.
Experience with automation frameworks and CI/CD pipelines.