Data Engineer (Python)

Role Overview: As a Python Developer with web scraping expertise, your main responsibility will be to design, develop, and maintain web scraping scripts using Python. You will be using web scraping libraries like Beautiful Soup, Scrapy, Selenium, and other tools to extract data from various websites. Your role will involve writing reusable, testable, and efficient code to extract both structured and unstructured data. Additionally, you will need to collaborate with software developers, data scientists, and other stakeholders to plan, design, develop, and launch new web scraping projects. Troubleshooting, debugging, and optimizing web scraping scripts will also be part of your responsibilities. Staying updated with the latest industry trends in automated data collection and cleaning is crucial, as well as maintaining code quality and organization of projects. Participation in code reviews to ensure alignment with standards and creating automated test cases for code functionality and performance will also be expected. Integration of data storage solutions like SQL/NoSQL databases, message brokers, and data streams for storing and analyzing scraped data will also fall under your purview. Key Responsibilities: - Design, develop, and maintain web scraping scripts using Python - Utilize web scraping libraries such as Beautiful Soup, Scrapy, Selenium, and other tools for data extraction - Write reusable, testable, and efficient code for structured and unstructured data extraction - Develop and maintain software documentation for web scraping scripts - Collaborate with software developers, data scientists, and stakeholders for new web scraping projects - Troubleshoot, debug, and optimize web scraping scripts - Stay updated with industry trends in automated data collection and cleaning - Maintain code quality and project organization - Participate in code reviews and ensure solutions align with standards - Create automated test cases for code functionality and performance - Integrate data storage solutions like SQL/NoSQL databases, message brokers, and data streams Qualifications Required: - Bachelor's degree in Computer Science, Software Engineering, or related field - 3-4 years of deep coding experience in Python - Experience with Python development and web scraping techniques - Familiarity with web frameworks like Django and Flask, as well as technologies such as SQL, Git, and Linux - Strong analytical and problem-solving skills - Good communication and teamwork abilities (Note: Job Location: Gurgaon, Work Model: Hybrid) Apply for this job or refer someone if you meet the required qualifications and skills.,