SENIOR DATA ENGINEER - Databricks

Bengaluru

Posted:1 week ago

Job Description

Data Engineer

 

Responsibilities:

  • Design, develop, and maintain scalable data pipelines using Databricks and Azure Data Factory.
  • Implement data processing solutions using Spark and Delta Tables to optimize performance and reliability.
  • Collaborate with data scientists and analysts to understand data requirements and deliver high-quality datasets.
  • Manage and optimize data storage solutions in Azure Data Lake and ensure data accessibility through Unity Catalog.
  • Monitor and troubleshoot data pipeline performance and implement improvements as necessary.
  • Document data processes and maintain clear communication with stakeholders regarding project status and challenges.
  • Stay updated with the latest industry trends and technologies to continuously improve data engineering practices.

Mandatory Skills:

  • Strong proficiency in SQL for data manipulation and querying.
  • Expertise in Python for data processing and automation.
  • Hands-on experience with Databricks is a deal breaker.
  • Familiarity with Azure Data Factory for orchestrating data workflows.
  • Experience with Spark for big data processing.
  • Knowledge of Delta Tables for efficient data storage and management.
  • Understanding of Unity Catalog for data governance and management.
  • Experience with Azure Data Lake for data storage solutions.
  • Familiarity with Azure Synapse for data integration and analytics.

Preferred Skills:

  • Experience with data modeling and ETL processes.
  • Knowledge of data warehousing concepts and best practices.
  • Familiarity with machine learning concepts and tools.
  • Experience with version control systems such as Git.
  • Strong problem-solving skills and attention to detail.

 

Skills Required

Azure Data Factory Data Processing Spark Azure Data Lake Data Pipeline Data Engineering SQL Data Manipulation Python Automation BIG Data Data Governance

Learn the skills this job asks for

See your skill gap for Data Engineer →

Free courses, interview questions and practice for each skill.

Job Type

Full Time

Mock Interview

Practice Video Interview with JobPe AI

Start Python Interview
cta

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

coding practice

Enhance Your Python Skills

Practice Python coding challenges to boost your skills

Start Practicing Python Now
Happiest Minds Technologies logo
Happiest Minds Technologies

IT Services and IT Consulting

Bengaluru Karnataka

RecommendedJobs for You