Staff Data Engineer

Sarvam AI

Bengaluru, IndiaPosted 4 months ago

Skill Required

EngineeringData EngineerObservabilityanalyticaldesigningsimilar)securitybuildingPostgreSQLEmbedded CAirflowTestNGGolangdesignSparkKafkaMySQLgRPCAPIsSQLGoAIFulltime

Key highlights

High-ownership, high-leverage role as technical owner of a foundational platform
5+ years of backend/data-platform engineering required
Must have ClickHouse, Golang, Kafka, and Debezium experience
On-site in Bengaluru

Role overview

Sarvam is building India's full-stack sovereign AI platform, focusing on research, models, infrastructure, and applications tailored for India. The company partners with leading enterprises and public institutions and is backed by prominent investors like Lightspeed, Peak XV, and Khosla Ventures. The role involves designing and building Sarvam's Data & Analytics Platform from scratch, with high ownership and technical responsibility. The platform will ingest events from all Sarvam products, replicate operational databases into an analytics store via CDC, and expose data through a tenant-aware query API and no-code dashboarding layer. The Staff Engineer will own the architecture, build core systems, set engineering standards, and shape the team around this platform.

Responsibilities

Design and build Sarvam's Data & Analytics Platform from the ground up as the technical owner
Ingest every meaningful event from every Sarvam product (e.g., outbound calls, agent turns, user turns, knowledge-base lookups, lead creation, deployments, model invocations) through a clean API layer that handles auto schema evolution and dynamic table creation per event type
Replicate product databases into the analytics store via CDC, ensuring operational data remains queryable alongside event streams without load on transactional systems
Expose data through a tenant-aware query API and a no-code dashboarding layer, powering customer-facing product analytics, internal BI, and the finance team's billing pipeline
Own architecture and build core systems hands-on, including databases, Kafka, and internal stores, ensuring all reads/writes go through APIs you design
Build a high-throughput event ingestion API in Go that accepts arbitrary event payloads, validates them, and lands them in ClickHouse with automatic table creation, column addition, and safe schema evolution
Develop CDC pipelines using Debezium and Kafka to mirror product Postgres/MySQL databases into ClickHouse, with deduplication, ordering guarantees, and replay tooling
Create a multi-tenant query API with strict RBAC, per-tenant isolation, query budgets, and a unified surface for customer dashboards, internal analytics, and finance billing extracts
Build a no-code dashboard layer enabling customers to create product-analytics views without SQL and allowing internal teams to ship dashboards quickly
Design and maintain the operational backbone: ClickHouse cluster (sharding, replication, MergeTree families, materialized views), capacity planning, cost controls, observability, on-call playbooks, and SLOs
Shape the engineering culture for the team, including design reviews, RFC processes, testing standards, and hiring benchmarks

Requirements

5+ years of backend/data-platform engineering experience, with significant time spent designing and operating analytics systems in production
Track record of building 0-to-1 platforms that other engineering teams depend on
ClickHouse in production at scale, including sharding, replication, MergeTree variants, materialized views, projection design, query tuning, and operational know-how (backups, upgrades, capacity planning)
Golang expertise for high-throughput backend services, with strong concurrency, performance, and API design (REST/gRPC) skills
Kafka as a primary streaming backbone, including partitioning strategy, consumer groups, exactly-once/idempotency patterns, and schema registry
Debezium-based CDC experience, including building and operating CDC pipelines from transactional databases into analytical stores
Strong systems instincts: reasoning about throughput, latency, cost, and failure modes before writing code

Nice to have

Spark + Airflow for batch enrichment, backfills, and scheduled aggregations
Experience building multi-tenant SaaS platforms with row-level security, per-tenant quotas, and tenant-aware RBAC
Worked on or integrated with BI/dashboarding tools (Superset, Metabase, Cube, or similar), especially embedded customer-facing analytics
Exposure to billing/metering data feeds where correctness is non-negotiable

Additional details

Work on-site in Bengaluru with a small team of senior builders, shipping fast
Databases, Kafka, and internal stores are never exposed directly — every read/write goes through APIs you design