Senior Manager

0 years

0 Lacs

Posted:2 days ago| Platform: Linkedin logo

Apply

Work Mode

On-site

Job Type

Full Time

Job Description

Job Summary

The Databricks Architect is responsible for designing, implementing, and optimizing scalable data solutions using the Databricks platform. This role involves working closely with cross-functional teams to deliver high-performance, secure, and reliable data architecture that support advanced analytics, machine learning, and business intelligence

Key Responsibilities

  • Design Databricks-based data architectures and models to meet business requirements
  • Develop event-driven and scalable GenAI application architectures
  • Refactor and rehost legacy applications onto cloud platforms
  • Design, develop, and optimize data pipelines and ETL processes using Azure Databricks (PySpark /Spark SQL).
  • Implement data security best practices and ensure compliance with industry standards
  • Collaborate with data architects, analysts, and other developers to deliver data solutions aligned with business requirements.
  • Translate business requirements into technical solutions
  • Perform data wrangling, cleansing, transformation, and aggregation from multiple sources.
  • Implement and maintain data lake and data warehouse solutions using Azure services (ADLS, Synapse, Delta Lake).
  • Monitor pipeline performance, troubleshoot issues, and ensure data integrity and reliability.
  • Develop and maintain CI/CD pipelines for Databricks workflows and jobs.
  • Participate in code reviews, unit testing, and documentation of data processes.

Required Skills & Experience

  • Strong experience with Databricks, Delta Lake, PySpark/Scala Spark, and Unity Catalog.
  • Proficiency in Python, SQL, and familiarity with relational and NoSQL databases
  • Experience with cloud platforms (Azure, AWS, GCP) and Infrastructure as Code tools
  • Knowledge of CI/CD, DevSecOps, Kafka, MLFlow, and Structured Streaming
  • Solid understanding of Azure Data Services: Azure Data Lake Storage (ADLS), Azure Data Factory (ADF)
  • Strong problem-solving skills and ability to work independently or as part of a team.
  • Databricks certifications (e.g., Certified Data Engineer, Associate Developer) preferred
  • Excellent communication skills.

Mock Interview

Practice Video Interview with JobPe AI

Start PySpark Interview
cta

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

Job Application AI Bot

Job Application AI Bot

Apply to 20+ Portals in one click

Download Now

Download the Mobile App

Instantly access job listings, apply easily, and track applications.

coding practice

Enhance Your Python Skills

Practice Python coding challenges to boost your skills

Start Practicing Python Now
EXL logo
EXL

Business Process Management / Analytics

New York

RecommendedJobs for You

hyderabad, telangana

gurugram, haryana, india

pune, maharashtra, india