Software Engineer, Distributed Data Systems (IN)
Onehouse.ai
IndiaremotePosted 8 days ago
Skill Required
EngineeringSoftware-EngineerDistributed-Systems-EngineeringData-Infrastructure-EngineeringBackend-EngineeringDistributed-Systems-EngineerDistributed-Database-EngineerSenior-Data-Infrastructure-EngineerSenior-Data-Management-EngineerRemote-Senior-Distributed-Systems-Engineer-Low-Latency-PipelinesSenior-Data-Management-Platform-EngineerDeveloperData EngineerSoftware EngineerSoftware DeveloperDesign SystemR ProgrammingKubernetesanalyticalData StructuresSnowflakeanalyticsbuildingQuery OptimizationdesignC++SparkLinuxCloudjQueryJavaAWSandAIMachine LearningFulltime
Key highlights
- Salary/compensation: Pay range above (base salary range or on-target earnings); dependent on location, experience, and other factors
- Level Required: Not explicitly stated beyond 'strong experience' with distributed systems and databases; PhD a nice-to-have but not required
- Key benefit: Equity Compensation; unlimited PTO (1 week mandatory); 26 weeks paid parental leave
- Notable requirement: Robust and clear communication skills; experience with inner workings of distributed multi-tiered systems, algorithms, and relational databases
Role overview
Onehouse is a mission-driven company dedicated to freeing data from data platform lock-in, delivering the industry's most interoperable data lakehouse through a cloud-native managed service built on Apache Hudi. The Data Infrastructure team serves as the bridge between open source and enterprise, contributing directly to Apache Hudi (used at scale by global enterprises) while building cornerstone infrastructure that solves complex optimization and systems problems. As a foundational member of this team, you will productionize the next generation of the data tech stack by building software and data features that process all ingested data.
Responsibilities
- Design new concurrency control and transactional capabilities that maximize throughput for competing writers
- Design and implement new indexing schemes, specifically optimized for incremental data processing and analytical query performance
- Design systems that help scale and streamline metadata and data access from different query/compute engines
- Solve hard optimization problems to improve the efficiency (increase performance and lower cost) of distributed data processing algorithms over a Kubernetes cluster
- Leverage data from existing systems to find inefficiencies, and quickly build and validate prototypes
- Collaborate with other engineers to implement and deploy, safely rollout the optimized solutions in production
- Productionize the next generation of our data tech stack by building the software and data features that actually process all of the data we ingest
- Accelerate our open source <> enterprise flywheel by working on the guts of Apache Hudi's transactional engine and optimizing it for diverse Onehouse customer workloads
- Act as a SME to deepen our teams' expertise on database internals, query engines, storage and/or stream processing
Requirements
- Strong, object-oriented design and coding skills (Java and/or C/C++ preferably on a UNIX or Linux platform)
- Experience with inner workings of distributed (multi-tiered) systems, algorithms, and relational databases
- You embrace ambiguous/undefined problems with an ability to think abstractly and articulate technical challenges and solutions
- An ability to prioritize across feature development and tech debt with urgency and speed
- An ability to solve complex programming/optimization problems
- An ability to quickly prototype optimization solutions and analyze large/complex data
- Robust and clear communication skills
Nice to have
- Experience working with database systems, Query Engines or Spark codebases
- Experience in optimization mathematics (linear programming, nonlinear optimization)
- Existing publications of optimizing large-scale data systems in top-tier distributed system conferences
- PhD degree with 2+ years industry experience in solving and delivering high-impact optimization projects
Benefits
- Equity Compensation; our success is your success with eligible participation in our company equity plan
- Health & Well-being; we'll invest in your physical and mental well-being by reimbursing up to 20,000 INR for your monthly insurance premium
- Financial Future; we'll invest in your financial well-being by making this role eligible for the provident fund of which Onehouse will contribute up to 1800 INR/month
- Location; we are a remote-friendly company (internationally distributed across N. America + India), though some roles will be subject to in-person requirements in alignment with the needs of the business
- Generous Time Off; unlimited PTO (mandatory 1 week/year minimum), uncapped sick days and 17 paid company holidays
- Food & Meal Allowance; weekly lunch stipend, in-office snacks/drinks
- Equipment; we'll provide you with the equipment you need to be successful and a one-time $500 (USD) stipend for your initial office/desk setup
- Child Bonding!; 26 weeks off for birthing and 12 weeks for surrogate and adoptive parents - fully paid so you can focus your energy on your newest addition
Additional details
- Onehouse is a $68M total funding company (backed by Craft, Greylock and Addition Ventures, with a fresh $35M Series B)
- Apache Hudi is already used at scale by global enterprises like Uber, Amazon, ByteDance etc
- Company values: One Team (optimize for company, team, self in that order), Tough & Persevering, Keep Making It Better Always, Think Big Act Fast, Be Customer Obsessed
- Pay Range Transparency: committed to fair and equitable compensation; job titles may span more than one career level; total compensation includes equity options and benefits listed
- Base pay range is subject to change and may be modified in the future
- Actual compensation packages dependent upon several factors: job-related skills, depth of transferable experience, relevant certifications and training, business needs, market demands and specific work location
- Riding off a fresh $35M Series B backed by Craft, Greylock and Addition Ventures
- Located in: internally distributed across N. America + India