Home
Jobs

Posted:3 months ago| Platform: Naukri logo

AI Match Score
Apply

Work Mode

Work from Office

Job Type

Full Time

Job Description

Design, develop, and maintain end-to-end data pipelines using Databricks and Apache Spark . Build and manage ETL processes for transforming and processing large datasets in real-time and batch modes. Define and govern data modelling and design standards, tools, and best practices, developing and maintaining data models and schemas to support analytics and machine learning applications. Collaborate with cross-functional teams, including data scientists, business analysts, and IT, to understand data requirements and deliver high-quality solutions. Optimize performance and ensure scalability of Databricks-based solutions, addressing data quality, processing time, and resource usage. Implement data governance best practices, ensuring data integrity, privacy, and security across all data assets. Monitor and troubleshoot pipeline performance, ensuring reliability and quick resolution of data issues. Automate data workflows and implement CI/CD pipelines for smooth deployment of data applications. Stay up-to-date with emerging technologies and best practices related to Databricks, Spark, and cloud data solutions. Requirements Required Qualifications: Bacheloror Masterdegree in Computer Science, Engineering, Data Science, or a related field. Proven experience as a Data Engineer with Databricks and Apache Spark . Strong programming skills in Python , Scala , or SQL . Hands-on experience working with cloud platforms such as AWS , Azure , or Google Cloud . Solid understanding of data warehousing , ETL , and data modelling principles. Experience with Databricks Delta Lake and building scalable data pipelines. Familiarity with Apache Kafka , Airflow , or other workflow automation tools. Knowledge of version control tools like Git and experience with continuous integration and deployment pipelines. Excellent analytical, data profiling, and problem-solving abilities Excellent oral and written communication

Mock Interview

Practice Video Interview with JobPe AI

Start Computer Science Interview Now
cta

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

Job Application AI Bot

Job Application AI Bot

Apply to 20+ Portals in one click

Download Now

Download the Mobile App

Instantly access job listings, apply easily, and track applications.

Mivada

IT Services and IT Consulting

Barangaroo New South Wales

51-200 Employees

15 Jobs

    Key People

  • John Doe

    CEO
  • Jane Smith

    CTO

RecommendedJobs for You