Staff Software Engineer- Search Quality
Databricks
Skill Required
Key highlights
- Role focuses on AI and human search optimization
- Works with hybrid retrieval (keyword + semantic)
- Requires IR/RAG expertise
- Inclusive hiring practices and diverse culture
Role overview
As a Search Quality Engineer, you will play a central role in democratizing Data and AI by building the contextual backbone for both AI agents and human users. Your mission is to ensure high-quality, accurate, and actionable search results—optimizing retrieval for LLMs to ground their reasoning in 'ground truth' data while also enhancing traditional search for intuitive, high-recall human experiences. This involves tackling advanced challenges like hybrid retrieval (balancing keyword and semantic search), dual-optimization for human and AI needs, and maintaining high-stakes accuracy to prevent AI hallucinations. You’ll work with heterogeneous data (structured SQL, unstructured docs, real-time metrics) and have a front-row seat to the Agentic revolution, solving some of the hardest problems in data discovery to power decision-making at scale.
Responsibilities
- Own the quality of search results for AI agents, optimizing the retrieval layer to enable LLMs to reason over untrained data and synthesize accurate, high-stakes business actions.
- Improve the traditional search experience for human users, ensuring employees can find assets and answers through intuitive, high-recall interfaces.
- Tackle hybrid retrieval, balancing traditional keyword-based search (for exactness) with semantic vector search (for intent).
- Fine-tune ranking models to satisfy both human readability and LLM-ready context (dual-optimization).
- Build guardrails and relevance scoring systems to ensure AI stays grounded in reality and prevent hallucinations.
- Connect and integrate data across structured SQL tables, unstructured documents, and real-time business metrics.
- Build evaluation frameworks (human-in-the-loop and LLM-based) to measure relevance.
- Work with relevance metrics such as nDCG, MRR, and Precision@K.
Requirements
- Expertise in Lucene/Elasticsearch, embeddings, and ranking algorithms.
- Strong understanding of Information Retrieval (IR) principles.
- Experience applying traditional IR knowledge to modern Retrieval-Augmented Generation (RAG) systems.
Nice to have
- Obsession with relevance metrics and quality measurement.
- Passion for solving complex data discovery challenges.
Benefits
- Comprehensive benefits and perks tailored to employee needs (region-specific details available via provided link).
Additional details
- Databricks is the Data and AI company, serving over 20,000 organizations worldwide, including 70% of the Fortune 500, with a unified platform (Genie, Lakebase, Agent Bricks, Lakeflow, Lakehouse, Unity Catalog).
- Headquartered in San Francisco with 30+ global offices.
- Databricks is committed to diversity and inclusion, ensuring hiring practices are inclusive and meet equal employment opportunity standards. All individuals are considered without regard to protected characteristics (age, color, disability, ethnicity, family/marital status, gender identity/expression, language, national origin, physical/mental ability, political affiliation, race, religion, sexual orientation, socio-economic status, veteran status, etc.).
- If the role requires access to export-controlled technology or source code, Databricks may apply for a U.S. government license at its discretion and may decline to proceed with an applicant on this basis.