Data Engineer

Valuelabs

3 - 5 years

3 - 8 Lacs

hyderabad

Posted:1 hour ago| Platform:

Apply

Skills Required

airflow pyspark azure snowflake python sql

Work Mode

Work from Office

Job Type

Full Time

Job Description

Position - Data Engineer

Experience - 3+ yrs

Location - Hyderabad

Notice Period - Immediate Joiners

Roles & Responsibilities:

3+ years of hands-on experience in data engineering with strong proficiency in:
Python & PySpark (for data transformation and pipeline development)
SQL (advanced querying,optimization, window functions, CTEs)
Spark (PySpark/Scala) for distributed data processing at scale
Data Warehouse Architecture (Snowflake, Azure Synapse, Redshift, BigQuery)
Cloud Platforms: Azure (ADLS, Data Factory, Databricks, Synapse) and AWS (S3, Glue, EMR, Lambda, Redshift)
DBT (for modeling, testing and documentation of data pipelines)
Airflow (for workflow orchestration and monitoring)
Hadoop Ecosystem (HDFS, Hive, YARN) experience with legacy systems is a plus.
Design, develop, and maintain high-performance, scalable ETL/ELT pipelines using Python, PySpark, and Scala on AWS and Azure platforms.
Architect and manage cloud-based data warehouses using Snowflake and Azure Synapse, ensuring optimal performance, cost-efficiency, and security.
Implement data transformation workflows using DBT (Data Build Tool) to enable clean, reusable and well-documented data models across business units.
Orchestrate complex data workflows using Apache Airflow, ensuring reliability, monitoring, and observability across pipelines.
Optimize data processing performance on Databricks and EMR using Spark (PySpark/Scala) for large-scale batch and streaming workloads.
Ensure data quality, lineage, governance, and compliance through integration with Purview, Datadog, or similar tools.
Automate CI/CD pipelines for data deployments using Azure DevOps, GitHub Actions, Jenkins, or GitLab CI.
Work closely with cross-functional teams to onboard new clients, validate data integrity and support data migration and modernization initiatives.
Mentor junior engineers and contribute to knowledge sharing through documentation, code reviews and training sessions.

More Jobs at Valuelabs

React JS Developer

Bengaluru

8 - 12 yrs

INR 25 - 35 Lacs

Business Analyst

Hyderabad

4 - 9 yrs

INR 0 - 3 Lacs

Fullstack Developer

Bengaluru

5 - 10 yrs

INR 15 - 30 Lacs

Api Automation Tester

Hyderabad

2 - 7 yrs

INR 5 - 12 Lacs

Senior Software Engineer-Node.js+React.js

Chennai

6 - 11 yrs

INR 22 - 32 Lacs

Mock Interview

Practice Video Interview with JobPe AI

Start PySpark Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.