Lead Azure Data Engineer

Immediate #HIRING

We’re a leading software company specializing in Artificial Intelligence, Machine Learning, Data Analytics, Innovative data solutions, Cloud-based technologies

t rizwana@randomtrees.com.

Overview

highly skilled Lead Data Engineer

hands-on problem solver

Key Responsibilities

• Design, develop, and optimize

  data pipelines

  using

  Azure and Databricks

  .
• Hands-on work with

  Databricks Asset Bundles (DABs)

  for CI/CD and environment management.
• Build and manage scalable

  ELT/ETL workflows

  with Databricks, Delta Lake, ADF, and other Azure services.
• Continuously adopt and apply

  new Databricks features

  to accelerate delivery.
• Optimize Spark, SQL, and Python code for

  performance and scalability

  .
• Apply

  software engineering best practices

  (e.g., TDD, CI/CD, modular design, code reviews) to ensure a

  robust, maintainable, and strategic data ecosystem

  .
• Mentor team members to

  improve overall team productivity and technical capability

  .
• Deliver in a

  fast-paced environment with quick turnaround expectations

  .

Required Skills & Experience

• 8+ years

  in Data Engineering with at least

  5 years hands-on in Azure & Databricks

  .
• Strong expertise in:

• Databricks

  (Delta Lake, Databricks SQL, Workflows,

  Databricks Asset Bundles

  , MLflow basics, unity catalog)

• Azure Data Services

  (ADF, Azure Synapse (good to have), Azure Data Lake, Azure SQL)

• Python, SQL, Spark

  (with proven performance optimization skills)
• Strong knowledge of

  modern data architectures

  (Lakehouse, governance).
• Track record of

  efficient delivery

  and helping teams accelerate execution.

Soft Skills & Attributes

• Efficiency-driven mindset

  – focused on speeding up both personal and team delivery.

• Strong communicator within the team

  , able to share knowledge and mentor effectively.

• Highly motivated, proactive, and results-oriented

  .
• Ability to

  work in a fast-paced environment with quick turnarounds

  .
• Team-first approach – drives

  collective improvement and efficiency

  .

Preferred Skills

• Experience with AI-driven development (e.g., Cursor.ai, GitHub Copilot, or similar tools) to accelerate coding, improve productivity, and maintain high-quality implementations.

• Exposure to Generative AI in Databricks

  (LLMOps, Model Serving, MosaicML, vector search) – good to have.

Work Hours

• Must have

  5–6 hours daily overlap with USA team (EST time zone)

  for collaboration.