Posted today · be early

L1 Data Engineer - Remote

DeepSource

IndiaremotePosted 1 day ago

Skill Required

Data-EngineeringCloud-Data-EngineeringETL-ELT-DevelopmentData-Pipeline-EngineeringAzure-Data-EngineeringRemote-Data-EngineerRemote-Mid-Level-Data-EngineerData-EngineerData EngineerQuery Optimizationdata engineeringAzureSQL ServerEngineeringDatabricksanalyticalautomationTerraformdesigninganalyticssecuritybuildingSparkPythonPandasdesignCI/CDCloudDesign PatternsjQueryETLSQLandCIFulltime

Key highlights

3+ years of professional Data Engineering experience.
Must hold or be actively working toward Databricks Certified Data Engineer Associate certification.
Hands‑on experience with Azure Data Factory and Databricks.
Proficiency in Python, Pandas, and PySpark for data processing.
Working knowledge of Azure Synapse Analytics (Spark pool integration).
Opportunity to deepen expertise in cloud‑native platforms (Microsoft Azure and Databricks).

Role overview

We are looking for a motivated and technically solid L1 Data Engineer to join our growing Data & Analytics team. In this role, you will be responsible for designing, building, and maintaining the data architecture and infrastructure that supports our organization's data strategy, and you will work hands‑on to develop, test, and deploy reliable data solutions ensuring pipelines are scalable, efficient, and aligned with business requirements. This is an ideal opportunity for a data professional eager to deepen expertise in cloud‑native data platforms, particularly within the Microsoft Azure and Databricks ecosystem, and who thrives in a collaborative, fast‑paced environment.

Responsibilities

Design, develop, and maintain scalable data pipelines and ETL/ELT workflows to support business intelligence and analytics use cases.
Build and optimize data ingestion processes using Azure Data Factory and Databricks, ensuring data quality and consistency across all layers of the data platform.
Transform and process large datasets using PySpark and Python, applying best practices for performance and maintainability.
Write and optimize complex SQL queries to support analytical reporting and data validation requirements.
Collaborate with data architects and senior engineers to implement and maintain data models aligned with organizational standards.
Monitor, troubleshoot, and resolve pipeline failures and data quality issues, applying root‑cause analysis to prevent recurrence.
Contribute to documentation of data pipelines, data dictionaries, and engineering standards.
Support the team in exploring and evaluating new tools and approaches to continuously improve the data infrastructure.

Requirements

3+ years of professional experience in a Data Engineering or closely related role.
Strong proficiency in Python for data processing, transformation, and automation tasks.
Hands‑on experience with Pandas for data manipulation and PySpark for distributed data processing.
Practical experience with Databricks, including notebook development, clusters, and job orchestration.
Experience building and managing data pipelines with Azure Data Factory.
Working knowledge of Azure Synapse Analytics, particularly Spark pool integration.
Solid SQL skills, including query writing, optimization, and performance tuning.
Familiarity with data engineering principles including incremental loading, data lake architecture, and Delta Lake.
Understanding of data governance and security concepts within a cloud data platform.
Candidates are expected to hold or be actively working toward the Databricks Certified Data Engineer Associate certification.

Nice to have

Experience with SQL Server migration projects, including schema conversion and data movement.
Exposure to Terraform for Azure infrastructure provisioning and management.
Familiarity with CI/CD practices applied to data engineering workflows.
Experience with Delta Sharing or Lakehouse Federation concepts.