Posted today · be early
Parallel Software Engineer
DataPelago
India, United StatesremotePosted today
Skill Required
Parallel-Software-EngineerGPU-Software-EngineerHigh-Performance-Computing-EngineerSystems-Software-EngineerSoftware-EngineerSoftware-DeveloperDistributed-Systems-EngineerSoftware-Systems-EngineerSoftware EngineerSoftware DeveloperSystem DesignQuery OptimizationMachine Learningtroubleshootingrelated fieldEngineeringR ProgramminganalyticaldevelopinganalyticssecuritybuildingTestNGdesignC++SparkFlinkLinuxCI/CDCloudGenerative AIjQueryandCICDFulltime
Key highlights
- Competitive compensation and stock options
- 5+ years experience (Bachelors) or 3+ years (Masters)
- Comprehensive benefits package
- Expertise in SIMD-extensions (AVX) or GPU programming (CUDA, ROCm)
Role overview
DataPelago is building an innovative data processing engine transforming how Apache Spark, Apache Flink, Ray and others operate on diverse, large-scale data. As a Parallel Software Engineer, you will be a key individual contributor developing advanced parallel software that unlocks the full potential of diverse hardware accelerators including GPUs and SIMD-capable CPUs, enhancing the functional breadth, performance, scale, and reliability of data processing operators.
Responsibilities
- Architect: Influence the architecture of how our data processing engine harnesses the parallelism in GPUs and SIMD-capable CPUs efficiently in processing diverse large-scale data.
- Design: Lead design of functional and performance enhancements to operators and functions that are accelerated by our engine.
- Core Development: Individually design, implement, test, optimize, and maintain parallel implementations of operators and functions on diverse acceleration hardware.
- Innovation and Differentiation: Analyze advances in accelerated computing hardware, programming models, and related tools and ensure our engine extends technology and product leadership.
- Collaboration: Partner effectively with the execution engine engineering team in integrating parallel software components with the overall engine.
- Continuous Improvement: Foster best practices in design and code reviews, testing, CI/CD, and issue resolution to maintain highest product quality, security, efficiency, & productivity.
Requirements
- Bachelor's degree in Computer Science or a related field with 5+ years of relevant experience OR a Master's degree in Computer Science or a related field with 3+ years of relevant experience.
- 3+ years of deep technical experience in developing production applications that process large scale data using SIMD-extensions of CPUs (e.g., AVX), GPU programming models (e.g., CUDA, ROCm), or equivalent accelerated computing framework.
- Demonstrated experience working with software libraries, development tools, and profiling tools specific to parallel and accelerated computing.
- Demonstrated experience in troubleshooting and resolving functional and performance anomalies in both pre- and post-production scenarios.
- Strong knowledge of computer architecture.
- Exceptional programming skills in C, C++.
- Extensive development experience in Linux environments.
- Strong analytical and problem-solving skills with a passion for performance optimization.
Benefits
- Technology Leadership: Shape the architecture and development of core parallel data processing functions of our engine.
- Cutting-Edge Innovation: Work on challenging problems at the forefront of accelerated computing and data processing.
- Significant Impact: Your contributions will directly impact the performance and scalability of our mission-critical platform.
- Growth: Expand your technical expertise and scope of responsibilities working with other talented engineers and with a growing product.
- Competitive compensation, stock options, comprehensive benefits package, leadership development opportunities.
Additional details
- Location: Mountain View, CA / Hyderabad, IN / Remote
- Location Considerations: We value face-to-face collaboration, but recognize that talent can be found anywhere. Our engineering team works at our headquarters in Mountain View, CA, at our India office in Hyderabad, and at remote locations.
- Originally posted on Himalayas
