Web Crawler Developer

Job Description

Job Description: We are seeking a creative and independent Web Crawler Developer to join our team Seattle based Construction Team. The ideal candidate will have a keen eye for detail, a passion for problem-solving, and the ability to think outside the box to develop sophisticated web scraping solutions. Responsibilities: - Design, implement, and maintain web crawlers that can effectively extract data from various websites . - Analyze web page structures and adapt crawlers to extract relevant information efficiently. - Monitor crawler performance and make necessary adjustments to ensure optimal data collection. - Work independently to identify new opportunities for data extraction and offer insightful recommendations. - Ensure compliance with legal and ethical standards for data scraping. - Collaborate with data analysts and other team members to understand data needs and improve data accuracy. - Keep up-to-date with the latest web scraping technologies and best practices Qualifications: - Strong experience with web scraping tools and frameworks (e.g., Scrapy, BeautifulSoup, Selenium, etc.). - Proficiency in programming languages such as Python, Java, or others relevant to web crawling. - Experience with handling and parsing different data formats like HTML, JSON, XML, etc. - Excellent problem-solving skills and the ability to think outside the box. - Ability to work independently and manage multiple tasks efficiently. - Solid understanding of web protocols (HTTP, HTTPS) and web technologies. - Familiarity with version control systems, preferably Git. - Knowledge of data privacy laws and ethical web scraping practices. Preferred: - Experience with cloud services like AWS or Azure for deploying and managing web crawlers. - Understanding of databases and data storage solutions. - Previous experience in a similar role or related projects. Job Type: Contractual / Temporary Contract length: 2 months Pay: ₹76,000.00 - ₹80,000.00 per month Benefits: Work from home Supplemental Pay: Performance bonus Expected Start Date: 03/06/2025