The Data Engineer will be an integral member of the IT Data Tribe of an Extended Team, collaborating on key data projects to migrate data pipelines and processes from legacy platforms to the Datahub. The role focuses on ensuring robustness and scalability while following BNP Paribas group guidelines and technologies.
Responsibilities
- Migrate the existing Hadoop infrastructure to cloud infrastructure on Kubernetes Engine, COS, Spark as a service, and Airflow as a service.
- Implement data transformation and quality to ensure data consistency and accuracy. Utilize programming languages such as Scala and SQL and tools like Spark for data transformation and enrichment operations.
- Set up CI/CD pipelines to automate deployments, unit testing and development management.
- Write and conduct unit and validation tests to ensure accuracy and integrity of code developed.
- Automate data pipelines and streamline data ingestion through the implementation of different orchestrators and scheduling processes (Airflow as a Service mainly).
- Writing technical documentation (specifications, operational documents) to ensure knowledge capitalization.
- Team player.
- Adhere to the standards and practices followed in the Project.
- Foster a culture of continuous learning and improvement within the team.
- Collaborate with cross‑functional teams to understand data requirements and deliver solutions.
Requirements
- At least 5 years of working experience in Data engineering.
- Working experience on Spark on Scala/Python/Java (any of these languages).
- Knowledge on Apache Airflow, Oozie or any other similar scheduling tools.
- Strong knowledge of SQL and NoSQL databases.
- Good exposure to CI/CD tools (Gitlab, Jenkins…).
- Knowledge of Kubernetes containerization.
- Integration experience with S3 storage/COS and parquet (and ORC) format.
- Hands‑on knowledge of Unix shell scripting.
- Design effective prompts to leverage Gen AI tools across IT domains (e.g., development, testing, data generation, documentation) during the development stage.
- Agile methodology exposure.
- Proactive mindset.
- Business / IT relationship (including IT OPS).
- Ability to understand, explain and support change.
- Ability to Deliver / Results driven.
- Ability to collaborate / Teamwork with data squads and business teams.
- Bachelor’s or Master’s degree in Engineering, Computer Science, Artificial Intelligence, Data Science or equivalent.
Nice to have
- Exposure to any of the data virtualization tools like Dremio.
- Exposure to Kafka, Elasticsearch, Kibana, HVault.
- Working knowledge of HDFS, Hadoop and Hive.
- Knowledge of banking products.
Additional details
- Job Title: Data Engineer
- Department: BNPP Personal finance
- Business Line/Function: BNP Paribas Personal Finance – consumer credit and banking as a service, with SET Iberia as an IT & Operations hub.
- Location: Chennai
- About BNP Paribas Group: European Union’s leading bank operating in 65 countries with nearly 185,000 employees, offering commercial, personal, investment & protection, and corporate banking services.
- About BNP Paribas India Solutions: Established in 2005, wholly owned subsidiary with delivery centers in Bengaluru, Chennai and Mumbai, serving corporate & institutional banking, investment solutions, and retail banking.
- Commitment to Diversity and Inclusion: The bank embraces diversity, prohibits discrimination and harassment, and promotes equal employment opportunity for all employees and applicants.
Zobhira
