Senior Data Engineer

Job description

Job Title: Senior Data Engineer
Location: Coimbatore
Experience: 5+ Years
Job Type: Full-Time

Key Responsibilities

• Design, develop, and maintain robust data pipelines using Airflow and AWS services.
• Implement and manage data warehousing using Databricks and PostgreSQL.
• Automate recurring tasks using Git and Jenkins.
• Build and optimize ETL processes leveraging AWS tools like S3, Lambda, AppFlow, and DMS.
• Create interactive dashboards and reports using Looker.
• Collaborate with various teams to ensure seamless integration of data infrastructure.
• Ensure the performance, reliability, and scalability of data systems.
• Use Jenkins for CI/CD and task automation.

Required Skills & Expertise

• Experience as a senior individual contributor on data-heavy projects.
• Strong command of building data pipelines using Python and PySpark.
• Expertise in relational database modeling, ideally with time-series data.
• Proficiency in AWS services such as S3, Lambda, and Airflow.
• Hands-on experience with SQL and database scripting.
• Familiarity with Databricks and ThoughtSpot.
• Experience using Jenkins for automation.

Nice to Have

• Proficiency in data analytics/BI tools such as Power BI, Tableau, Looker, or ThoughtSpot.
• Experience with AWS Glue, AppFlow, and data transfer services.
• Exposure to Terraform for infrastructure-as-code.
• Experience in data quality testing.
• Previous interaction with U.S.-based stakeholders.
• Strong ability to work independently and lead tasks effectively.

Qualifications

• Bachelor’s or Master’s degree in Computer Science, Data Engineering, or related field.
• 5+ years of relevant experience.

Tech Stack

• Databricks
• PostgreSQL
• Python & PySpark
• AWS Stack (S3, Lambda, Airflow, DMS, etc.)
• Power BI / Tableau / Looker / ThoughtSpot
• Git / Jenkns / CI-CD tools

Job Type: Full-time

Pay: ₹500,000.00 - ₹2,500,000.00 per year

Ability to commute/relocate:

• Coimbatore, Tamil Nadu: Reliably commute or planning to relocate before starting work (Required)

Experience:

• Data Engineer: 6 years (Required)

Work Location: In person

Job Types: Full-time, Permanent

Pay: ₹500,000.00 - ₹2,500,000.00 per year

Ability to commute/relocate:

• Coimbatore West, Coimbatore, Tamil Nadu: Reliably commute or planning to relocate before starting work (Required)

Education:

• Master's (Required)

Experience:

• Data Engineer: 6 years (Required)