Senior Member, Data Engineering

As a member of the Data Engineering team, your primary responsibility will be to handle various aspects of data extraction. This includes understanding the data requirements of the business group, reverse-engineering the website and its technology, developing web robots to automate data extraction, and building monitoring systems to ensure data integrity. You will also play a key role in managing changes to website dynamics and layout for clean downloads, creating scraping and parsing systems to structure raw data, and providing operations support for high availability and zero data losses. Moreover, you will be involved in storing extracted data in recommended databases, constructing scalable data extraction systems, and automating data pipelines. The ideal candidate for this role should possess the following qualifications: - 2-4 years of experience in website data extraction and scraping. - Proficiency in relational databases, including writing complex SQL queries and ETL operations. - Strong command of Python for data operations. - Expertise in Python frameworks such as Requests, UrlLib2, Selenium, Beautiful Soup, and Scrapy. - Familiarity with HTTP requests and responses, HTML, CSS, XML, JSON, and JavaScript. - Skill in using debugging tools in Chrome for reverse engineering website dynamics. - Academic background in BCA/MCA/BS/MS with a solid foundation in data structures and algorithms. - Strong problem-solving, analytical, and debugging skills. If you meet these qualifications and are enthusiastic about working in a dynamic data engineering environment, we encourage you to apply for this position.,