Web Scraping Engineer

As an experienced automation engineer, your role involves designing, building, and maintaining sophisticated web scraping systems and autonomous application agents. You will combine technical expertise in web automation with strategic thinking to develop tools that efficiently collect data and interact with web applications. - Design and develop robust web scraping architectures capable of collecting structured data from complex websites, including those with anti-bot measures. - Build intelligent agents that can autonomously navigate job application workflows, including form completion and document submission. - Implement sophisticated rate limiting, proxying, and browser fingerprinting techniques to ensure reliable operation. - Create data processing pipelines to clean, normalize, and enrich collected information. - Develop monitoring systems to detect and adapt to website changes. - Build user-friendly interfaces for controlling automation systems. To excel in this role, you should have: - 4+ years of experience with web scraping and automation technologies. - Strong programming skills in Python and JavaScript. - Advanced experience with browser automation libraries such as Puppeteer, Playwright, or Selenium. - Expertise in parsing HTML/DOM structures and handling dynamic content (AJAX, SPAs). - Experience implementing IP rotation, user-agent cycling, and browser fingerprinting solutions. - Knowledge of data storage solutions for organizing and querying large datasets. - Familiarity with headless browser environments, containerization, and deploying automation scripts in Lambda AWS. Preferred qualifications include: - Experience with proxy management services and residential IP networks. - Knowledge of machine learning techniques for solving CAPTCHAs and pattern recognition. - Experience with natural language processing for enhancing form completion logic. - Familiarity with ethical and legal considerations surrounding web automation. - Experience developing against enterprise authentication systems. - Understanding of job application workflows and ATS (Applicant Tracking Systems). In return, the company offers a competitive salary and benefits package, a flexible work environment, cutting-edge technical challenges, and the opportunity to shape the development of next-generation automation systems.,