Data Engineer

Location:

Chennai & Hyderabad

Workplace Type:

Hybrid (4 days in a week)

Shift Timings

: 1 PM to 10 PM

Notice Period

: Immediate to 25 Days

About The Role

We are seeking a highly skilled and motivated Data Engineer to join our growing data team. In this role, you will be responsible for designing, building, and maintaining robust and scalable data pipelines that power our analytics and business intelligence initiatives. You will work with a variety of data sources and technologies, ensuring data quality, performance, and reliability. The ideal candidate will have a strong background in data warehousing, ETL processes, and cloud computing, with a passion for solving complex data challenges and delivering impactful solutions. You will collaborate closely with data scientists, analysts, and business stakeholders to understand their data needs and provide them with the data they need to make informed decisions. This is an excellent opportunity to contribute to a data-driven culture and make a significant impact on our organization.

Key Responsibilities

• Design, build, and maintain robust, scalable data ingestion and ETL workflows using tools such as Python, PySpark, SQL, and Snowflake
• Ingest and process structured and unstructured data from diverse sources into data lakes/warehouses
• Optimize ETL jobs for speed, efficiency, and fault tolerance while ensuring high data quality
• Work closely with data scientists, analysts, and business teams to ensure timely and accurate delivery of datasets
• Implement and maintain data quality monitoring and alerting systems
• Develop and maintain data documentation and metadata
• Troubleshoot and resolve data pipeline issues
• Participate in code reviews and contribute to the development of best practices
• Stay up-to-date with the latest data engineering technologies and trends
• Automate data integration processes to improve efficiency and reduce manual effort
  

Required Skills & Qualifications

• Bachelor's degree in Computer Science, Engineering, or a related field
• 5+ years of experience in data engineering or a similar role
• Strong proficiency in Python and PySpark
• Expertise in SQL and data warehousing concepts
• Experience with Snowflake or other cloud-based data warehouses
• Experience with data integration tools and techniques
• Experience with Jenkins CI/CD for deployment
• Experience with scheduling tools such as Autosys
• Experience with cloud platforms such as AWS
• Strong understanding of data modeling and data architecture principles
• Excellent problem-solving and analytical skills
• Strong communication and collaboration skills
  

Additional Information

Experience with Ataccama is a plus. This role offers a competitive salary and benefits package, as well as opportunities for professional development and growth. We are an equal opportunity employer and value diversity at our company. We do not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status. The position is hybrid, requiring a minimum of 3 days per week in the New York office to foster collaboration and team cohesion. We encourage candidates from all backgrounds to apply.