Data and AI Lead, Finance
Databricks
Bengaluru, IndiaPosted 1 month ago
Skill Required
FinanceAIGitHub ActionsData WarehousingCD pipelinesEngineeringDatabricksSalesforcesimilar)securitybuildingSparkTestNGPythondesignGitCI/CDUnityCloudETLSQLGenerative AIManual TestingCICDMachine Learning
Key highlights
- Based in Bengaluru
- 12+ years of experience required
- High-impact role at intersection of finance and data platform
- Technical lead and mentor role
- Comprehensive benefits package
- SOX compliance requirements
Role overview
This role is a part of the Finance Organisation. As Finance Data Lead on the Finance Data and AI team, you will be the technical authority behind the data pipelines, AI systems, and internal applications that power Databricks' Finance and Accounting organisation. You will report to the Senior Manager, Finance Data and AI and serve as the individual contributor who sets the technical bar, drives architectural decisions, and delivers the highest-complexity work on the team. This is a high-impact role at the intersection of finance domain expertise and data platform capability. You will work closely with a team of finance data engineers as a technical lead and mentor, and partner directly with Accounting, FP&A, and Finance leadership to define what gets built and how. This individual is expected to be based in Bengaluru.
Responsibilities
- Design and develop ETL pipelines using Databricks SQL and Python / PySpark to enhance reporting, automate journal entries, and transform core financial processes across various domains in accounting and FP&A, such as revenue, expenses, equity, commissions, and tax
- Architect, build, and own the most complex finance data pipelines in the Finance data lake using Databricks Jobs and Lakeflow Declarative Pipelines with built-in data validation and reconciliations
- Lead the technical design of AI use cases for the Finance and Accounting organisation, including forecasting automation, anomaly detection, and natural language interfaces to financial data
- Build and maintain internal Finance applications (Databricks Apps, Genie Agents, AI/BI dashboards) that enable self-service for non-technical Finance stakeholders
- Design and deliver curated Finance datasets and enforce row-level security, data access policies, and Unity Catalog governance standards
- Define and champion coding standards, data modelling conventions, documentation practices, and testing frameworks across the Finance engineering team
- Enforce and evolve Git-based version control, pull-request review processes, and CI/CD pipelines (Declarative Automation Bundles, GitHub Actions) to satisfy SOX change management requirements
- Lead technical scoping and solutioning for requirements from Accounting, FP&A, Internal Audit, and Procurement teams
- Serve as the primary technical point of contact during financial close, ensuring data accuracy and timely resolution of pipeline issues
- Partner with IT and Engineering on new system integrations, providing detailed technical requirements and leading UAT
- Mentor junior and mid-level engineers through code reviews, pairing, and design discussions, raising the technical quality of the broader team
- Proactively identify architectural debt, performance bottlenecks, and tooling gaps, and drive resolution with minimal direction
Requirements
- 12+ years of experience in data engineering, analytics engineering, or finance systems, with a track record of owning complex, production-grade pipelines end-to-end
- Deep proficiency in SQL and Python; hands-on experience with Apache Spark and the Databricks platform
- Experience building and maintaining ELT/ETL pipelines from financial source systems (NetSuite, Salesforce, Stripe, Zuora, or similar) into a centralised data lake
- Strong understanding of core finance and accounting concepts, including close processes, revenue recognition, intercompany, chart of accounts, and financial reporting
- Ability to independently translate ambiguous Finance requirements into well-architected, maintainable technical solutions
- Comfortable driving technical conversations with both engineering peers and non-technical Finance stakeholders
- Experience building Finance-facing dashboards, self-service BI products, and executive reporting layers
- Familiarity with AI/ML concepts with demonstrated enthusiasm for applying them to Finance workflows
Nice to have
- Prior experience at a high-growth SaaS or cloud infrastructure company
- Hands-on experience with AI/BI tools, Genie, or LLM-powered applications
- Experience with Declarative Automation Bundles or CI/CD for Finance DataLake pipelines
- CPA, CFA, or formal finance/accounting background
Benefits
- Comprehensive benefits and perks that meet the needs of all employees (region-specific details available via provided link)
Additional details
- Role ID: GAQ327R175
- Reports to: Senior Manager, Finance Data and AI
- Location: Bengaluru
- Databricks is the Data and AI company, serving over 20,000 organizations worldwide, including 70% of the Fortune 500. Platform includes Genie, Lakebase, Agent Bricks, Lakeflow, Lakehouse, and Unity Catalog.
- Databricks is committed to fostering a diverse and inclusive culture where everyone can excel, with hiring practices that meet equal employment opportunity standards.
- If access to export-controlled technology or source code is required, Employer may apply for a U.S. government license at its discretion or decline to proceed with an applicant on this basis alone.