Hybrid
Full Time
Role: Data Engineer Location: Hyderabad Role Overview: The Azure Big Data Engineer is a hands-on technical role focused on designing, implementing, and maintaining robust, scalable data pipelines and storage solutions on the Azure platform. The engineer plays a critical role in enabling data-driven decision-making across the organization by building and managing high-performance data infrastructure. Key Responsibilities: Data Engineering & Pipeline Development Develop, test, and maintain ETL/ELT pipelines using tools such as Azure Data Factory, Azure Databricks, Synapse Analytics, and Microsoft Fabric. Ingest, wrangle, transform, and join data from various sources ensuring data quality and consistency. Implement data storage solutions using Azure Data Lake Storage, Lakehouse, Delta Lake, and data warehousing technologies (e.g., Synapse, Azure SQL Database). Performance Optimization & Monitoring Optimize data pipelines for cost, performance, scalability, and reliability. Monitor data flows and proactively troubleshoot pipeline and performance issues. Apply best practices for performance tuning and infrastructure alignment. Security & Compliance Implement data security measures including RBAC, data encryption, and compliance with organizational policies and regulatory standards. Address data governance requirements and audit readiness. Collaboration & Stakeholder Communication Work closely with data scientists, architects, and analysts to gather requirements and deliver solutions aligned with business needs. Translate business requirements into technical designs and documentation. Present architecture and design options to stakeholders and conduct technical demos. Expected Outcomes: Delivery & Execution Code pipelines and solutions following best practices in scalability, performance, and maintainability. Complete documentation for architecture, source-target mappings, test cases, and performance benchmarks. Perform unit testing, debugging, and performance validation of data pipelines. Quality & Process Adherence Ensure adherence to coding standards, project timelines, SLAs, and compliance protocols. Quickly resolve production bugs and reduce recurring issues through RCA. Improve pipeline efficiency (e.g., faster run times, reduced costs). Knowledge & Certifications Maintain up-to-date technical certifications and training. Contribute to reusable documentation, knowledge bases, and process improvements. Skills & Expertise: Technical Skills Programming: Proficiency in SQL, Python, and PySpark; familiarity with Scala. ETL Tools: Azure Data Factory, Azure Databricks, Microsoft Fabric, Synapse Analytics, Informatica, Glue, DataProc. Cloud Platforms: Expertise in Azure (including ADLS, Key Vault, Azure SQL, Synapse), familiarity with AWS or GCP a plus. Data Warehousing: Snowflake, BigQuery, Delta Lake, Lakehouse. Data Modeling: Strong understanding of dimensional modeling, schema design, and optimization for large datasets. Security: Knowledge of data security and compliance best practices. Soft Skills Strong analytical and troubleshooting skills. Excellent communication and collaboration capabilities. Ability to estimate and manage workload effectively. Customer-oriented approach with a focus on value delivery. Preferred Qualifications: Experience with Microsoft Fabric and modern data lakehouse architectures. Exposure to ML/AI concepts and integration with data pipelines. Azure certifications (e.g., Azure Data Engineer Associate, Azure Solutions Architect). Performance Metrics (KPIs): Adherence to engineering standards and timelines. Pipeline performance (run time, resource usage). Reduction in post-release defects and production incidents. Time to resolution for pipeline issues. Completion of certifications and mandatory training. Number of reusable assets created/shared. Compliance with data governance and security policies. Certifications: Azure Data Engineer Associate (preferred) Relevant domain certifications in data engineering, cloud, or big data technologies Tools & Technologies: ETL & Orchestration: Azure Data Factory, Databricks, Apache Airflow, Glue, Talend Cloud Services: Azure Synapse, ADLS, Azure SQL, Key Vault, Microsoft Fabric Programming: Python, SQL, PySpark, Scala (optional) Data Platforms: Snowflake, BigQuery, Delta Lake, Azure Lakehouse
UST
Upload Resume
Drag or click to upload
Your data is secure with us, protected by advanced encryption.
Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.
We have sent an OTP to your contact. Please enter it below to verify.
Instantly access job listings, apply easily, and track applications.
Hyderabad
11.0 - 12.0 Lacs P.A.
Hyderabad
Experience: Not specified
12.0 - 13.0 Lacs P.A.
Bengaluru
25.0 - 30.0 Lacs P.A.
Bengaluru
25.0 - 30.0 Lacs P.A.
25.0 - 30.0 Lacs P.A.
25.0 - 30.0 Lacs P.A.
Bengaluru
4.0 - 7.0 Lacs P.A.
Chennai
10.0 - 20.0 Lacs P.A.
Bengaluru
15.0 - 25.0 Lacs P.A.
6.0 - 16.0 Lacs P.A.
