Data bricks SME Engineer

Job Title: Data bricks SME Engineer

Work Timing: US EST Hours

Experience: 8+ years

Location: Remote

Job Responsibilities:

• Databricks Hands on SME Engineer Architect, configure, and optimize Databricks Pipelines for large-scale data processing within an Azure Data Lakehouse environment.
• Set up and manage Azure infrastructure components including Databricks Workspaces, Azure Containers (AKS/ACI), Storage Accounts, and Networking.
• Design and implement a monitoring and observability framework using tools like Azure Monitor, Log Analytics, and Prometheus/Grafana.
• Collaborate with platform and data engineering teams to enable micro services-based architecture for scalable and modular data solutions.
• Drive automation and CI/CD practices using Terraform, ARM templates, and GitHub Actions/Azure DevOps.

Required Skills & Experience:

• Strong hands-on experience with Azure Data bricks, Delta Lake, and Apache Spark.
• Deep understanding of Azure services: Resource Manager, AKS, ACR, Key Vault, and Networking.
• Proven experience in micro services architecture and container orchestration.
• Expertise in infrastructure-as-code, scripting (Python, Bash), and DevOps tooling.
• Familiarity with data governance, security, and cost optimization in cloud environments.

Bonus:

• Experience with event-driven architectures (Kafka/Event Grid).
• Knowledge of data mesh principles and distributed data ownership.

dsingh15@fcsltd.com