6 years
10 - 20 Lacs
Posted:1 day ago|
Platform:
On-site
Full Time
About The Opportunity Operating in the high-growth Data Engineering & Cloud Analytics sector, we help enterprises modernize their data landscapes using lakehouse architectures and advanced analytics. From financial services to retail, our engineering teams deliver scalable, secure, and insight-ready data platforms that power mission-critical decisions across India and overseas. Role: Data Engineer – PySpark & Databricks Location On-site, India Role & Responsibilities Design, build, and optimize large-scale batch and streaming data pipelines with PySpark on Databricks. Create and manage Delta Lake tables, schema evolution, and data quality constraints to guarantee ACID compliance and governance. Automate end-to-end ETL workflows with Azure Data Factory, Databricks Jobs, and scheduling triggers. Tune Spark configurations for cost, throughput, and sub-minute latency; implement real-time monitoring, alerting, and troubleshooting dashboards. Partner with Data Scientists, Analysts, and Product Owners to expose curated, high-quality datasets for AI, BI, and reporting use-cases. Lead coding standards, CI/CD pipelines, and security best practices, mentoring junior engineers on engineering excellence. Skills & Qualifications Must-Have 3–6 years’ hands-on experience with PySpark and Spark SQL in production environments. Expertise in Databricks notebooks, clusters, jobs orchestration, and Delta Lake optimisation. Strong command of Azure Data services—ADLS Gen2, Azure Data Factory, and Azure SQL/Synapse. Advanced SQL skills with proven data modelling, partitioning, and performance-tuning know-how. Fluency with Git-centric workflows and automated CI/CD (Azure DevOps, Jenkins, or GitHub Actions). Preferred Exposure to Structured Streaming, Kafka, or Event Hubs for real-time ingestion. Experience with Infra-as-Code (Terraform/ARM) and automated cluster deployment. Knowledge of MLflow, Unity Catalog, or data governance frameworks in lakehouse setups. Benefits & Culture Highlights Opportunity to work on greenfield cloud data platforms leveraging the latest Databricks features. Collaborative, learning-oriented culture with sponsored certifications and tech talks. Competitive salary, performance bonuses, and comprehensive medical coverage. Skills: sql,azure sql,azure data services,terraform,pyspark,kafka,spark sql,adls gen2,cloud storage,azure data factory,git,delta lake,structured streaming,performance tuning,azure synapse,databricks,ci/cd,data modeling
Viraaj HR Solutions Private Limited
Upload Resume
Drag or click to upload
Your data is secure with us, protected by advanced encryption.
Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.
We have sent an OTP to your contact. Please enter it below to verify.
Instantly access job listings, apply easily, and track applications.
Hyderabad, Telangana, India
10.0 - 20.0 Lacs P.A.
Hyderabad, Telangana, India
10.0 - 20.0 Lacs P.A.
