Job
Description
About The Role
Project Role :Custom Software Engineer
Project Role Description :Develop custom software solutions to design, code, and enhance components across systems or applications. Use modern frameworks and agile practices to deliver scalable, high-performing solutions tailored to specific business needs.
Must have skills :Microsoft Azure Analytics Services, Microsoft SQL Server, Microsoft Azure Databricks, PySpark, Azure Data Factory, Databricks REST API
Good to have skills :NA
Minimum 2 year(s) of experience is required
Educational Qualification :15 years full time education
Summary:About The Role :Azure Data Engineer – ADF, Databricks, ADLSRole OverviewWe are seeking a skilled and detail-oriented Azure Data Engineer to join our data engineering and governance team. The ideal candidate will have hands-on experience in designing, developing, and maintaining scalable data pipelines using Azure Data Factory, Azure Databricks, and Azure Data Lake Storage (ADLS). This role involves working closely with cross-functional teams to deliver robust data solutions that support analytics, reporting, and initiatives. Roles & Responsibilities:-Design, build, and maintain ETL/ELT pipelines using Azure Data Factory and Databricks. -Develop workflows that ingest, transform, and load data from diverse sources into ADLS, ensuring data quality and consistency. -Implement Bronze, Silver, and Gold layer transformations in Databricks using PySpark and SQL. -Optimize pipeline performance and cost efficiency through monitoring and tuning.-Collaborate with data analysts, and business stakeholders to understand data requirements and deliver solutions.-Maintain metadata, lineage, and audit trails for all data processes.-Manage access controls and security across Databricks workspaces and ADLS folders. -Document architecture, data flows, and transformation logic for compliance and future reference.-Support Power BI integration by enabling reporting layers and sandbox environments within Databricks.
Professional & Technical Skills:
Must To Have
Skills:Proficiency in Microsoft Azure Analytics Services, Microsoft Azure Databricks, PySpark, Microsoft SQL Server.-5+ years of experience in Azure Data Factory, Azure Databricks, and ADLS Gen2.-Strong proficiency in SQL, PySpark, and Spark-SQL.-Experience with Databricks REST API for workflow orchestration.-Understanding of data governance, security (RBAC, Azure AD), and compliance frameworks.-Exposure to data modeling, incremental transformations, and data mart integration.Nice to Have -Knowledge of open-source lakehouse formats like Delta Lake, Iceberg, and Parquet.-Experience with machine learning workflows and AI model deployment in Databricks.-Familiarity with Agile/Scrum methodologies and DevOps practices.Work Environment -Multiple Databricks workspaces including production sandboxes -Collaboration with ADLS and Databricks teams for access provisioning and data governance.-Integration with SAP HANA and other enterprise systems for data ingestion.
Additional Information:The candidate should have minimum 2 years of experience in Microsoft Azure Analytics Services.This position is based at our Pune office.A 15 years full time education is required.
Qualification 15 years full time education
