In2IT Technologies - Lead Data Engineer - ETL/Spark

Job Title :

Location :

Position Summary

Key Responsibilities

• Lead the design and development of complex SQL queries, views, stored procedures, and data models across large datasets.
• Optimize query performance, conduct SQL tuning, and manage indexing strategies for Azure SQL, DBSQL, and NoSQL systems.
• Build and maintain SQL-driven ETL pipelines using Databricks, Spark, and Python for batch
  

• Design and implement data warehouse solutions that enable analytical insights and real-time decision-making.
• Integrate and transform customer data from over 30 diverse sources using SQL and Python-based loaders.
• Govern, document, and enforce best practices for SQL code versioning, QA, and deployment workflows.
• Collaborate with business teams to define and build reusable data marts, data models, and analytical views.
• Lead and mentor junior engineers in SQL scripting, troubleshooting, and architectural best practices.
  

Key Skills & Technologies

• Advanced SQL development - complex joins, subqueries, window functions, CTEs, query optimization
• Azure SQL, Databricks SQL (DBSQL), NoSQL (MongoDB), Data Warehousing
• Spark SQL and Structured Streaming for real-time data ingestion
• ETL frameworks and data pipeline orchestration in Python
• Hands-on with Microsoft Customer Insight, Google Analytics, and Azure Data Services
• Data Governance, Data Strategy, and Customer Data Management
  

Qualifications & Experience

• Graduation in Computer Science or equivalent + 5 Years work experience
• Knowledge of Machine learning and AI is preferable
• Experience with CI/CD for SQL workflows, data quality frameworks, and SQL-based test automation
• Designing SQL solutions for data enrichment, audience segmentation, and real-time reporting
• Working experience in high-scale, consumer-focused environments