Design, implement, and optimize end-to-end data pipelines for ingesting, processing, and transforming large volumes of structured and unstructured data.
Develop data pipelines to extract and transform data in near real-time using cloud-native technologies.
Implement data validation and quality checks to ensure accuracy and consistency.
Monitor system performance, troubleshoot issues, and implement optimizations to enhance reliability and
efficiency
.
Collaborate with business users, analysts, and other stakeholders to understand data requirements and deliver tailored solutions.
Document technical designs, workflows, and best practices to facilitate knowledge sharing and maintain system documentation.
Provide technical guidance and support to team members and stakeholders as needed.

Desirable Competencies:

8+ years of work experience.
Proficiency in writing complex SQL queries on MPP systems (Snowflake/Redshift).
Experience in Databricks and Delta tables.
Data engineering experience with Spark/Scala/Python.
Experience in Microsoft Azure stack (Azure Storage Accounts, Data Factory, and Databricks).
Experience in Azure DevOps and CI/CD pipelines.
Working knowledge of Python.
Comfortable participating in 2-week sprint development cycles.

More Jobs at PHOTON

Cloud Engineer

Chennai, Tamil Nadu, India

5.0 - 8.0 yrs

INR 5 - 8 Lacs

Sr Data Architect

Chennai, Tamil Nadu, India

8.0 - 13.0 yrs

INR 8 - 13 Lacs

Mobile Developer

Chennai, Tamil Nadu, India

3.0 - 7.0 yrs

INR 3 - 7 Lacs

Data Engineer

Hyderabad, Telangana, India

3.0 - 7.0 yrs

INR 3 - 7 Lacs

Technical Lead_Offshore

Hyderabad, Telangana, India

10.0 - 12.0 yrs

INR 3 - 10 Lacs

Mock Interview

Practice Video Interview with JobPe AI

Start Job-Specific Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.