Job Title: Databricks Data Engineer 
 
 
 Experience Level: 3 to 6 Years
 Location: Ahmedabad
 Employment Type: Full-Time
 Certifications Required:
  - Databricks Certified Data Engineer Associate
  - Databricks Certified Data Engineer Professional
  - Cloud Certifications (Preferred): Azure, AWS, GCP
  
   Job Summary: 
We are seeking a highly skilled and certified Databricks Data Engineer to join our dynamic data engineering team. The ideal candidate will have hands-on experience in implementing Lakehouse architectures, upgrading to Unity Catalog, and building robust data ingestion pipelines. This role demands proficiency in Python, PySpark, SQL, and Scala, along with a strong understanding of big data technologies, streaming workflows, and multi-cloud environments.
   Key Responsibilities: 
Lakehouse Implementation:
  - Design and implement scalable Lakehouse architecture using Databricks.
  - Optimize data storage and retrieval strategies for performance and cost-efficiency.
  
 Unity Catalog Upgrade:
  - Lead and execute Unity Catalog upgrades across Databricks workspaces.
  - Ensure secure and compliant data governance and access control.
  
 Data Ingestion & Migration:
  - Develop and maintain data ingestion pipelines from various sources (structured, semi-structured, unstructured).
  - Perform large-scale data migrations across cloud platforms and environments.
  
 Pipeline Development:
  - Build and manage ETL/ELT pipelines using PySpark and SQL.
  - Ensure data quality, reliability, and performance across workflows.
  
 Big Data Streaming & Workflows:
  - Implement real-time data streaming solutions using Spark Structured Streaming or similar technologies.
  - Design workflow orchestration using tools like Airflow, Databricks Workflows, or equivalent.
  
 Multi-Cloud Expertise (Preferred):
  - Work across Azure, AWS, and GCP environments.
  - Understand cloud-native services and integration patterns for data engineering.
  
 Collaboration & Documentation:
  - Collaborate with data scientists, analysts, and business stakeholders to understand data requirements.
  - Document technical designs, data flows, and operational procedures.
  
   Required Skills & Qualifications: 
 - 3 6 years of hands-on experience in data engineering roles.
  - Strong expertise in Databricks platform and ecosystem.
  - Proficiency in Python, PySpark, SQL, and Scala.
  - Experience with Lakehouse architecture and Unity Catalog.
  - Proven track record in building scalable data ingestion pipelines and performing data migrations.
  - Familiarity with big data technologies such as Delta Lake, Apache Spark, Kafka, etc.
  - Understanding of data lake concepts and best practices.
  - Experience with streaming data and workflow orchestration.
  - Certified as Databricks Data Engineer Associate and Professional (mandatory).
  - Cloud certifications in Azure, AWS, or GCP are a strong plus.
  - Excellent problem-solving skills and attention to detail.
  - Strong communication and collaboration abilities.
  
   Nice to Have: 
 - Experience with CI/CD pipelines and DevOps practices in data engineering.
  - Exposure to data cataloging tools and metadata management.
  - Knowledge of data security, privacy, and compliance standards.
  
   Why Join Us
 - Work on cutting-edge data engineering projects in a fast-paced environment.
  - Collaborate with a team of passionate professionals.
  - Opportunity to grow and expand your skills across multiple cloud platforms.
  - Competitive compensation and benefits.