10 - 18 years

3 - 7 Lacs

Posted:1 week ago| Platform: Naukri logo

Apply

Work Mode

Work from Office

Job Type

Full Time

Job Description

Job Summary:

We are looking for an experienced Azure Databricks Engineer with expertise in SQL, data migration, and on-premises to cloud transition. The ideal candidate will work on designing, developing, and deploying data solutions using Azure Databricks in combination with SQL-based querying. You will play a critical role in migrating data and workloads from on-premises to Azure cloud environments.

Key Responsibilities:

  • Data Engineering with Azure Databricks:
    • Design, develop, and manage data pipelines and ETL processes in Azure Databricks.
    • Implement and optimize performance of Spark-based applications.
    • Write optimized SQL queries for querying data and performing analytics.
    • Develop and deploy interactive notebooks in Azure Databricks using SQL and PySpark.
  • Cloud Migration (On-premises to Cloud):
    • Lead the effort in migrating data from on-premises SQL servers and databases to Azure Cloud solutions.
    • Implement data integration strategies for seamless data migration from on-prem systems (SQL Server, Oracle, etc.) to Azure SQL databases and Data Lakes.
    • Assist with troubleshooting and resolving migration issues, ensuring minimal downtime and data loss.
  • SQL Querying and Data Transformation:
    • Use SQL-based solutions to transform, aggregate, and join large datasets from various sources.
    • Design and develop SQL queries for data extraction, transformation, and loading into the cloud platform.
    • Implement best practices for managing data quality, ensuring accurate and consistent data post-migration.
  • Collaboration and Support:
    • Work closely with cross-functional teams to understand data requirements and implement the best cloud-based solutions.
    • Provide ongoing support and training to team members on Databricks and migration strategies.
    • Continuously monitor the performance of Databricks jobs and SQL queries, identifying and resolving any performance bottlenecks.

Required Skills and Experience:

  • Proven Experience in Azure Databricks: Hands-on experience with Azure Databricks (using Spark, SQL, and Python).
  • SQL Proficiency: Strong command over SQL, including advanced queries, optimization techniques, and working with large datasets.
  • On-premises to Cloud Migration: Experience migrating data and workloads from on-premises environments to Azure Cloud (e.g., Azure SQL Database, Azure Data Lake, Azure Synapse Analytics).
  • Data Engineering Skills: Experience designing and implementing ETL pipelines and data workflows in Azure Cloud environments.
  • Cloud Data Platforms: Familiarity with other Azure tools like Azure Data Factory, Azure SQL, Azure Synapse Analytics, and Azure Blob Storage.
  • Data Transformation: Expertise in using SQL and Spark for data transformation, data cleaning, and ensuring data integrity.
  • Scripting: Experience with Python or Scala for automating processes and manipulating large datasets in Databricks.

Desired Skills:

  • Data Warehousing: Experience with Data Warehousing concepts and tools like Azure Synapse.
  • DevOps and CI/CD: Familiarity with CI/CD pipelines and automation in a cloud environment.
  • Azure Certification: Certifications such as Azure Data Engineer Associate, Azure Fundamentals, or Azure Databricks Certified Associate are a plus.
  • Agile Methodology: Comfortable working in an Agile environment with frequent iteration and communication.

Educational Qualifications:

  • Bachelors or Master's degree in Computer Science, Information Technology, or a related field, or equivalent experience.

Mock Interview

Practice Video Interview with JobPe AI

Start PySpark Interview
cta

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

Job Application AI Bot

Job Application AI Bot

Apply to 20+ Portals in one click

Download Now

Download the Mobile App

Instantly access job listings, apply easily, and track applications.

coding practice

Enhance Your Python Skills

Practice Python coding challenges to boost your skills

Start Practicing Python Now

RecommendedJobs for You

noida, mumbai, pune, chennai, bengaluru

gurugram, chennai, bengaluru