Data Engineer

Value1

IndiaremotePosted 12 days ago

Skill Required

Data-EngineerBig-Data-EngineerData-Platform-EngineerETL-DeveloperData-Integration-EngineerData-Engineer-JobsData-EngineeringData-Engineering-AnalystData-Engineering-SpecialistData-Engineering-JobsData-Engineering-PositionsData EngineerQuery Optimizationdata engineeringETLInformaticaSQL ServerEngineeringR ProgrammingCassandraanalyticsBigQueryRedshiftsecuritybuildingPostgreSQLAirflowMongoDBwrittenPythonHadoopOracledesignScalaSparkKafkaFlinkFulltime

Key highlights

Level Required: 3.5+ years of IT experience with 1.5+ years in data-related technologies
Must-have qualification: Hands-on experience with the Hadoop stack (HDFS, Kafka, Spark, Flink, etc.)
Preferred programming language: Java, Scala, or Python (Java preferred)

Role overview

Publicis Sapient is a global professional services company that enables clients to thrive in the digital era through expert strategies, customer-centric experience design, and world-class product engineering. With over 20 years in IT, they recognize a critical need for transformation across industries including finance, automotive, consumer products, retail, energy, and travel. They seek thought leaders and rock stars to drive innovation and help businesses improve daily lives.

Responsibilities

Perform technical design and implement components for data engineering solutions
Utilize deep understanding of data integration and big data design principles to create custom solutions or implement package solutions
Independently drive design discussions to ensure the necessary health of the overall solution
Build functionality for data ingestion from multiple heterogeneous sources in batch and real-time
Build functionality for data analytics, search, and aggregation
Design, develop, and deliver solutions covering data ingestion, integration, transformation, data storage and computation frameworks, performance optimizations, analytics and visualizations, infrastructure and cloud computing, and data management platforms

Requirements

Overall 3.5+ years of IT experience with 1.5+ years in data-related technologies
Minimum 1.5 years of experience in Big Data technologies
Hands-on experience with the Hadoop stack including HDFS, sqoop, kafka, Pulsar, NiFi, Spark, Spark Streaming, Flink, Storm, hive, oozie, and airflow
Strong experience in at least one of the programming languages Java, Scala, or Python (Java preferred)

Nice to have

Good knowledge of traditional ETL tools (e.g., Informatica, Talend) and database technologies (Oracle, MySQL, SQL Server, Postgres) with hands-on experience
Knowledge of data governance processes (security, lineage, catalog) and tools like Collibra or Alation
Knowledge of distributed messaging frameworks (ActiveMQ, RabbitMQ, Solace), search & indexing, and microservices architectures
Performance tuning and optimization of data pipelines
CI/CD infrastructure provisioning on cloud platforms, auto-build and deployment pipelines, and code quality practices
Working knowledge with data platform services on at least one cloud platform, IAM, and data security
Cloud data specialty and related Big data technology certifications