Azure Data Engineer

2 - 8 years

0 Lacs

Posted:3 hours ago| Platform: Linkedin logo

Apply

Work Mode

On-site

Job Type

Full Time

Job Description

Dear Candidates,


We’re Hiring! 🚀


Job Title:

Experience:

Location:

Employment Type:

Azure Data Engineer

Key Responsibilities:

1) Design and Development:

  • Design, develop, and deploy scalable data pipelines using

    Databricks (PySpark, Spark SQL)

    ,

    Azure Data Factory

    , and other Azure data services.
  • Implement ETL/ELT processes to ingest, transform, and load data from various sources into data lakes and data warehouses.
  • Optimize and tune data pipelines for performance and scalability.

2) Data Processing:

  • Write and optimize complex SQL queries for data extraction, transformation, and analysis.
  • Use

    PySpark

    for large-scale data processing and analytics.
  • Implement data partitioning, bucketing, and indexing strategies for efficient data retrieval.

3) Data Integration:

  • Integrate data from multiple sources, including structured, semi-structured, and unstructured data.
  • Work with APIs, streaming data, and batch processing to ensure seamless data integration.

4) Data Governance and Quality:

  • Implement data governance practices to ensure data quality, consistency, and security.
  • Monitor and troubleshoot data pipelines to ensure data accuracy and availability.

5) Collaboration:

  • Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions.
  • Work closely with DevOps teams to deploy and monitor data pipelines in production environments.

6) Documentation:

  • Document data pipelines, workflows, and processes for knowledge sharing and future reference.
  • Maintain up-to-date documentation on data architecture and data models.

Must-Have Skills:

  • Databricks:

    Hands-on experience with Databricks for data processing, analytics, and Serverless SQLWH, Unity Catalog, Lakehouse, Medallion Architecture
  • Azure Data Services:

    Proficiency in Azure Data Factory, Azure Data Lake Storage, and Azure SQL Database, Key Vault, Azure Pricing Model
  • SQL:

    Strong expertise in writing and optimizing complex SQL queries.
  • PySpark:

    Experience in using PySpark for data processing and transformation.
  • ETL/ELT:

    Strong understanding of ETL/ELT processes and tools.
  • Data Modeling:

    Knowledge of data modeling techniques and best practices.
  • Data Governance:

    Familiarity with data governance, data quality, and data security practices.

Good-to-Have Skills:

  • Experience with

    Azure DevOps

    for CI/CD pipelines.
  • Knowledge in

    DLT (Delta Live Table

    ) in databricks

Qualifications:

  • Bachelor’s or Master’s degree in Computer Science, Information Technology, or a related field.
  • 2-8 years of experience in data engineering, with a focus on Azure and Databricks.
  • Relevant certifications such as

    Microsoft Certified: Azure Data Engineer Associate

    or

    Databricks Certified Associate Developer

    are a plus.

Soft Skills:

  • Strong problem-solving and analytical skills.
  • Excellent communication and collaboration skills.
  • Ability to work in a fast-paced, dynamic environment.
  • Self-motivated with a strong sense of ownership and accountability




If you are interested in the job opportunity,

📩 Apply here - Azure Data Engineer - Banglore, PAN India – Fill out form

Mock Interview

Practice Video Interview with JobPe AI

Start PySpark Interview
cta

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

Job Application AI Bot

Job Application AI Bot

Apply to 20+ Portals in one click

Download Now

Download the Mobile App

Instantly access job listings, apply easily, and track applications.

coding practice

Enhance Your Skills

Practice coding challenges to boost your skills

Start Practicing Now
Deloitte logo
Deloitte

Professional Services

New York

RecommendedJobs for You

bengaluru, karnataka, india

pune, maharashtra, india