3.0 - 7.0 years

0 Lacs

pune, maharashtra

On-site

Agivant is looking for a skilled and dedicated Senior Data Engineer to join the expanding data team. As a Senior Data Engineer at Agivant, you will have a vital role in constructing and expanding our data infrastructure to facilitate data-informed decision-making throughout the organization. Your primary responsibilities will include designing, developing, and managing efficient and dependable data pipelines for both ELT (Extract, Load, Transform) and ETL (Extract, Transform, Load) processes. Your duties will involve collaborating with stakeholders to comprehend data requirements and transforming them into effective data models and pipelines. Additionally, you will be tasked with constructing and fine-tuning data pipelines utilizing a range of technologies such as Elastic Search, AWS S3, Snowflake, and NFS. Furthermore, you will be responsible for establishing and managing data warehouse schemas and ETL/ELT processes to cater to business intelligence and analytical needs. It will be your responsibility to introduce data quality checks and monitoring mechanisms to ensure data integrity and identify potential issues. You will also work closely with data scientists and analysts to guarantee data accessibility and usability for various analytical purposes. Staying informed about industry best practices, CI/CD/DevSecFinOps, Scrum, and emerging data engineering technologies is also essential. Your contribution to advancing and refining our data warehouse architecture will be invaluable. ### Responsibilities: - Design, develop, and maintain robust and scalable data pipelines for ELT and ETL processes, ensuring data accuracy, completeness, and timeliness. - Collaborate with stakeholders to understand data requirements and translate them into efficient data models and pipelines. - Build and optimize data pipelines using technologies like Elastic Search, AWS S3, Snowflake, and NFS. - Develop and maintain data warehouse schemas and ETL/ELT processes to support business intelligence and analytics needs. - Implement data quality checks and monitoring to ensure data integrity and identify potential issues. - Work closely with data scientists and analysts to ensure data accessibility and usability for analytical purposes. - Stay current with industry best practices, CI/CD/DevSecFinOps, Scrum, and emerging data engineering technologies. - Contribute to the development and enhancement of our data warehouse architecture. ### Requirements: #### Mandatory: - Bachelor's degree in Computer Science, Engineering, or a related field. - 5+ years of experience as a Data Engineer focusing on ELT/ETL processes. - At least 3+ years of experience in Snowflake data warehousing technologies. - At least 3+ years of experience in creating and maintaining Airflow ETL pipelines. - Minimum 3+ years of professional-level experience with Python for data manipulation and automation. - Working experience with Elastic Search and its application in data pipelines. - Proficiency in SQL and experience with data modeling techniques. - Strong understanding of cloud-based data storage solutions like AWS S3. - Experience working with NFS and other file storage systems. - Excellent problem-solving and analytical skills. - Strong communication and collaboration skills.,

Posted 3 weeks ago

Apply

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.