FBS - Elasticsearch Data Engineer (Medallion Architecture)
Capgemini Technology Services
IndiaremotePosted 1 month ago
C
Skill Required
Elasticsearch-Data-EngineerData-EngineeringBig-Data-EngineeringMedallion-ArchitectureELK-Stack-EngineerElasticsearch-EngineerData-EngineerData EngineerElasticsearchQuery Optimizationdata engineeringAirflowETLSpring BootEngineeringServerlessR ProgrammingKubernetesanalyticalJenkinswrittenPythonLookerdesignSparkCloudjQueryJavadbtAWSIAMandAIFulltime
Key highlights
- Competitive salary and performance‑based bonuses.
- Minimum 5 years of Elasticsearch/ELK experience.
- Flexible work arrangements (remote or office).
- Strong expertise required in Spark and Amazon EMR.
- BS in Computer Science or related field required.
- Full English fluency required.
Role overview
Our client is one of the United States’ largest insurers, serving more than 10 million households with over 19 million individual policies across all 50 states. With gross written premiums exceeding US$25 billion, the company operates within one of the world’s largest insurance groups and employs nearly 18,500 people. The Data Engineer will architect, develop, and maintain scalable data pipelines within a medallion architecture (bronze, silver/base vault, business vault, gold layers), enabling high‑quality, business‑ready datasets by leveraging modern data engineering technologies and orchestration practices.
Responsibilities
- Design, build, and manage end‑to‑end data pipelines across the medallion architecture—specifically the bronze, silver (base vault with DBT and orchestration tools, business vault), and gold layers.
- Ingest and process raw data using Spark and Amazon EMR for scalable, distributed computation.
- Develop and automate data transformations for the base vault using DBT (Data Build Tool) to standardize and model data efficiently.
Requirements
- At least 5 years of experience as an Elasticsearch Data Engineer – ELK (Elasticsearch, Logstash, Kibana) stack expert.
- Java Spring Boot experience.
- IBM ACE Programming experience.
- BS in Computer Science, Data Engineering (Big Data, AWS certification), Data Modeling or a similar field.
- Full English fluency.
- Strong understanding of data modeling, governance, and best practices in modern data architectures.
- Excellent analytical, problem‑solving, and communication skills.
- Elasticsearch – cluster optimization, query development, data modeling, performance tuning & administration (4‑6 years).
- Deep experience with Spark, Python and ETLs and Amazon EMR.
- Hands‑on experience with DBT for data transformation and modeling.
- Apache Airflow, AWS Step Functions, or similar orchestration tools.
- Expert knowledge of Amazon S3 and Apache Iceberg for data storage and management.
- AWS Cloud – intermediate level (3‑4 years) including Lambda, Step Functions, IAM, SNS, API Gateway, VPC, Transit Gateway.
- JSON – intermediate (4‑6 years).
- Jenkins – Data Pipeline intermediate (4‑6 years).
- CloudWatch – intermediate (4‑6 years).
Nice to have
- Experience with Kubernetes for container orchestration.
- Experience with Dremio, Looker, or equivalent business view/semantic layer technologies.
- Jenkins – Data Pipeline intermediate (4‑6 years) PLUS (additional depth).
- CloudWatch – intermediate (4‑6 years) PLUS (additional depth).
Benefits
- Competitive salary and performance‑based bonuses.
- Comprehensive benefits package.
- Career development and training opportunities.
- Flexible work arrangements (remote and/or office‑based).
- Dynamic and inclusive work culture within a globally renowned group.
- Private health insurance.
- Pension plan.
- Paid time off.
- Training & development programs.
Additional details
- Capgemini is a global leader in partnering with companies to transform and manage their business through technology.
- Capgemini has over 340,000 team members in more than 50 countries and generated €22.5 billion in revenue in 2023.
- The posting was originally on Himalayas.
- The client is part of one of the largest insurance groups in the world.