Senior Data Engineer

Key Responsibilities

• Design and implement scalable, reliable, and secure data architectures.
• Develop best practices and frameworks for data integration, governance, and security.
• Define and drive the roadmap for enterprise data solutions aligned with business objectives.
• Architect, build, and optimize ETL/ELT pipelines using Azure Data Factory (ADF), Databricks, and PySpark.
• Work with SQL and big data technologies to ensure efficient data processing and storage.
• Design and manage data lakes, data warehouses, and cloud-based storage solutions.
• Implement cloud-based data solutions, primarily using Microsoft Azure (Azure Data Factory, Azure Synapse, Azure Databricks).
• Ensure cost-effective and high-performance cloud solutions for data processing and analytics.
• Lead and mentor a team of data engineers, ensuring adherence to best practices.
• Collaborate with stakeholders, including data analysts, business leaders, and software engineers, to define and implement data-driven solutions.
• Provide technical guidance and architectural direction for complex projects.
• Ensure high availability, scalability, and performance of data systems.
• Implement data security policies and ensure compliance with industry standards (GDPR, HIPAA, etc.).
• Optimize data storage and retrieval for maximum efficiency and cost savings.

Requirements

• Proven experience in designing and implementing large-scale data solutions.

Technical Skills

• SQL: Advanced skills in query optimization and database management.
• PySpark: Strong experience in data transformation and processing.
• Databricks: Hands-on experience in big data frameworks and analytics.
• Azure Data Factory (ADF): Expertise in orchestrating ETL/ELT pipelines.
• Cloud Platforms: Strong experience with Azure (preferred), AWS, or GCP.
• Data Warehousing: Deep understanding of data modeling, data lakes, and data warehouse architectures.

Job Type: Full-time

Pay: ₹470,561.62 - ₹1,816,006.73 per year

Work Location: In person