Hire22.ai logo

Data Engineer

Hire22.ai

3 - 7 years

12 - 20 Lacs

India

Posted:2 days ago

Job Description

Our client, is looking for an experienced Data Engineer to design, develop, and maintain scalable data solutions, build efficient data pipelines, manage data integration processes, and ensure data quality and performance.


Experience 3-7 years



RESPONSIBILITIES


• Pipelines & Ingestion: Independently build database, file, and API ingestion into ADLS Gen2 and Azure Databricks; implement incremental loads, change data capture where appropriate, and safe reprocessing.


• Transformations & Modeling: Develop SQL and Python/PySpark transformations and Delta Lake tables; build bronze, silver, and gold layers and analytics-ready models with clear data contracts.


• Reliability & Orchestration: Schedule dependencies through Databricks Jobs or Azure Data Factory; implement validation, monitoring, alerts, retries, and recovery procedures to meet agreed data freshness targets.


• Security & Delivery: Apply Unity Catalog access controls and lineage, use Azure Key Vault or approved secret management, and deliver automated tests and CI/CD releases through Git-based review.


• Performance & Partnership: Tune Spark jobs, queries, and table layout for performance and cost; gather requirements with U.S.-based stakeholders and document mappings, quality rules, and operational runbooks.


• AI-Assisted Engineering: Use approved AI tools to assist with SQL/Python development, testing, troubleshooting, and documentation; validate generated code and outputs and protect client data.


REQUIREMENTS


Experience ranges are indicative; equivalent demonstrated capability will be considered.


• Required - Experience: Typical range: 3-5 years in data engineering, including 2+ years delivering Azure Databricks solutions. Demonstrate independent pipeline delivery from requirements through deployment and production support.


• Required - Core Skills: Strong SQL, Python/PySpark, Delta Lake, ADLS Gen2, ETL/ELT, and data modeling; hands-on incremental ingestion, orchestration, automated testing, and Spark performance tuning.


• Required - Delivery: Practical Unity Catalog, secure Azure access, Git/CI/CD, monitoring, and recovery experience; ability to manage commitments and explain data quality issues to U.S.-based stakeholders.


• Preferred - Additional Platforms: Experience with one or more: Azure Data Factory, Event Hubs, Structured Streaming, Lakeflow pipelines, Terraform, Microsoft Fabric, or Azure Synapse Analytics. Depth across all tools is not expected.


• Preferred - Integration & Certification: Power BI integration or analytics data modeling is a plus. Relevant Databricks data engineering certification is preferred; equivalent practical experience is welcomed.


Skills Required

Data Integration Data Quality Azure Databricks SQL Python Pyspark Analytics Azure Data Factory Azure Ci/cd GIT Spark

Job Type

Full Time

Mock Interview

Practice Video Interview with JobPe AI

Start Python Interview
cta

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

coding practice

Enhance Your Python Skills

Practice Python coding challenges to boost your skills

Start Practicing Python Now
Hire22.ai logo
Hire22.ai

RecommendedJobs for You

Meghaz Software Solutions logo

Data Engineer

Meghaz Software Solutions

Bengaluru

Independent Recruitment Consultant (freelancer) logo

Data Engineer

Independent Recruitment Consultant (freelancer)

India

People Tech Technology logo

Data Engineer

People Tech Technology

Hyderabad

WPP logo

Chennai

Krato Solutions logo

Data Engineer

Krato Solutions

Chennai

Applied Systems logo

Data Engineer

Applied Systems

Bengaluru

HP logo

Bengaluru