Data Engineer
Avahi
IndiaremotePosted 1 month ago
Skill Required
Data-EngineeringCloud-Data-EngineeringData-Platform-EngineeringAWS-Data-EngineeringBig-Data-EngineeringData-EngineerData-Engineer-JobsData-Engineering-JobsData-Engineering-PositionsData EngineerQuery OptimizationAWSObservabilityMicrosoft DynamicsWeb AccessibilityEngineeringSalesforceanalyticalElasticsearchCryptographyTerraformdesigningRedshiftDynamoDBsecurityWorkdayHubSpotPythonHadoopdesignSparkFlinkCloudDesign PatternsHiveFulltime
Key highlights
- Open to candidates based in India only
- Remote-first work environment
- Focus on AWS data platforms and cloud-first solutions
- Career development and mentorship opportunities
- Commitment to diversity, inclusion, and accessibility
Role overview
At Avahi, a premier cloud-first consulting company, this role focuses on designing, building, and maintaining scalable AWS data platforms to support batch/streaming pipelines, analytics, and AI/ML workloads. The position is remote-first and open exclusively to candidates based in India, emphasizing collaboration, innovation, and customer-centric solutions in a diverse, global team environment.
Responsibilities
- Design, build, and maintain scalable AWS data platforms supporting batch and streaming pipelines, analytics, and AI/ML workloads, aligned with AWS Well-Architected best practices.
- Build and operate data ingestion, transformation, and enrichment pipelines from internal systems and external APIs, handling structured, semi-structured, unstructured, and graph data.
- Implement data normalization workflows to ensure consistent schemas, high data quality, and reliable analytics, BI, and ML use cases.
- Design and enforce data governance including cataloging, lineage, access control, and auditability.
- Build and maintain knowledge graphs to model relationships across core business entities, enabling advanced analytics and inference.
- Identify data gaps, inconsistencies, and missing relationships using strong analytical and inference skills.
- Integrate data from enterprise platforms such as CRM and ERP systems (Salesforce, HubSpot, SAP, NetSuite, Dynamics 365, Workday).
- Design secure data access layers for analytics, BI, ML, and downstream applications.
- Implement monitoring, observability, and data quality checks for freshness, completeness, and pipeline health.
- Optimize data architectures for performance and cost efficiency using partitioning, indexing, compression, and storage tiering.
- Build internal tooling, dashboards, and standardized scaffolding to improve visibility, maintainability, and onboarding.
- Collaborate with cross-functional teams to deliver high-impact data solutions and share best practices, documentation, and technical guidance.
Requirements
- Strong experience designing and operating AWS data platforms, including S3, Glue, Lake Formation, Athena, Redshift, EMR, Kinesis/MSK, DynamoDB, OpenSearch, and Neptune.
- Strong Python skills for data engineering, focused on modular, testable, and maintainable code.
- Solid understanding of distributed data systems, including batch and streaming pipelines, fault tolerance, idempotency, and event-driven architectures.
- Experience with data warehouse and lakehouse architectures, ETL/ELT pipelines, and analytical query engines.
- Hands-on experience with Spark, Hadoop, Hive, or Flink.
- Strong data modeling skills, including normalized, denormalized, and graph-based models, with safe schema evolution.
- Advanced SQL skills for analytics and data engineering, including window functions, CTEs, and query optimization.
- Experience integrating external APIs and enterprise systems, especially CRM and ERP platforms.
- Knowledge of data governance, security, and compliance, including encryption, access control, and audit logging.
- Experience implementing monitoring, observability, and data quality checks using CloudWatch and CloudTrail.
- Comfort with Infrastructure as Code using CloudFormation or Terraform.
- Strong end-to-end ownership mindset, with a focus on scalability, reliability, and long-term maintainability.
- Professional-level English communication skills, able to explain data architectures and trade-offs to technical and non-technical stakeholders.
- Candidates must be based in India; applications from outside India will not be considered.
Benefits
- Remote-First Flexibility: Enjoy work-life harmony in a remote-first environment that allows you to work from anywhere.
- Innovative Culture: Embrace a startup mindset, encouraging creativity, agility, and growth.
- Career Development: Access mentorship and opportunities to advance your career.
- Purpose-Driven Mission: Join a company dedicated to championing diversity, supporting women in tech, and fostering sustainable practices.
- Global Collaboration: Work alongside a diverse, talented team, sharing insights and collaborating on innovative solutions.
Additional details
- Company Overview: Avahi is a cloud-first consulting company specializing in Managed Services, Reselling, Staffing, and Professional Services, with a focus on customer-centricity, collaboration, agility, innovation, integrity, and diversity & inclusion.
- Accessibility and Inclusivity Statement: Avahi is committed to fostering a workplace that celebrates diversity and inclusivity, welcoming applicants from all backgrounds. The company is an equal opportunity employer and provides accommodations during the recruitment process upon request.
- Originally posted on Himalayas.