Spark Developer

InfraCloud

IndiaremotePosted 18 days ago

Skill Required

Spark-DeveloperBig-Data-EngineeringDistributed-Systems-EngineeringSoftware-EngineerApache-Spark-DevelopmentApache-Spark-DeveloperPySpark-DeveloperSpark-DevelopmentPySpark-DevelopmentSenior-Java-Spark-DeveloperDatabricks-DeveloperApache-SparkSparkSystem DesignR ProgrammingKubernetesdebugginganalyticsbuildingdesignScalaCloudjQueryJavaandFulltime

Key highlights

Senior Software Engineer level
Strong experience with Apache Spark internals and distributed systems required
Requires hands-on experience contributing to open-source projects
Role involves contributing to the Apache Spark open-source community

Role overview

We are looking for a Senior Software Engineer with strong expertise in Apache Spark and distributed systems. The role involves working on Spark internals, contributing to the Apache Spark open-source community, and building scalable data processing solutions aligned with the platform requirements.

Responsibilities

Contribute fixes and enhancements to the Apache Spark open-source project.
Upgrade and maintain compatibility with newer Spark releases.
Optimize Spark internals for performance, scalability, and reliability.
Design and develop distributed data processing solutions.
Debug complex Spark-related performance and stability issues.
Collaborate with product and platform teams on architecture and technical initiatives.

Requirements

Strong experience with Apache Spark internals and distributed systems.
Hands-on experience contributing to open-source projects.
Strong Java and/or Scala programming skills.
Expertise in query execution, distributed processing, and performance optimization.
Strong debugging and problem-solving abilities.
Ability to work independently and drive technical initiatives.